2023-09-28 23:24:53,017 - utils - INFO - Namespace(attack_ratio=0.4, attack_type='None', batch_size=[100, 100], beta=1, data_dir='data', dataset='adult', delta_g=0.5, delta_l=0.5, device='cuda', device_id='2', disparity_type='TPSD', drop_last=False, eps=[0.01, 0.03, 0.005], eps_delta=[0.01, 0.01], eps_delta_g=0.01, eps_delta_l=0.01, eps_g=0.01, eps_vg=0.005, eps_vl=0.03, eval_epoch=20, factor_delta=0.1, force_active=True, global_epoch=0, lam=1, local_epochs=5, log_dir='results/EFFL_adult', log_name='log', lr_delta=0.1, max_epoch_stage=[750, 750, 500], method='EFFL', n_clients=2, n_feats=94, n_hiddens=20, norm='loss+', num_workers=0, q=6.0, s=1, sampler='None', seed=3, sensitive_attr='race', shuffle=True, step_size=0.03, target_dir_name='results/EFFL_adult', test_dir='data/adult/test', theta=1, train_dir='data/adult/train', weight_fair=1.0)
2023-09-28 23:24:54,688 - utils - INFO - stage1_gradient_single_runtime: 0.008420944213867188
2023-09-28 23:24:54,689 - utils - INFO -  epoch: 0, all client loss: [0.7432671189308167, 0.6731126308441162], all pred client disparities: [0.02399306371808052, 0.011154040694236755], all client disparities: [0.0, 0.0016611367464065552], all client accs: [0.2590799033641815, 0.6599166393280029],  alpha_performance: tensor([0.4462, 0.5538], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:54,836 - utils - INFO - valid: True, epoch: 0, loss: [0.737181544303894, 0.6707944273948669], accuracy: [0.3093922734260559, 0.6719875931739807], mean_accuracy:0.4906899333000183,variance_accuracy:0.1812976598739624, disparity: [0.0, 0.016234450042247772], mean_disparity:0.008117225021123886,variance_disparity:0.008117225021123886, pred_disparity: [0.04235217720270157, 0.00128936767578125]
2023-09-28 23:24:54,919 - utils - INFO - global_valid: True, epoch: 0,  global_loss: 0.6715324521064758, global_accuracy: 0.45268405589460414,  global_disparity:0.015146806836128235, global_pred_disparity: 0.0012032687664031982,
2023-09-28 23:24:55,247 - utils - INFO - stage1_gradient_single_runtime: 0.002479076385498047
2023-09-28 23:24:55,248 - utils - INFO -  epoch: 1, all client loss: [0.7459393739700317, 0.670466959476471], all pred client disparities: [0.021023813635110855, 0.009031042456626892], all client disparities: [0.0, 0.002475433051586151], all client accs: [0.2590799033641815, 0.6731056571006775],  alpha_performance: tensor([0.4262, 0.5738], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:55,505 - utils - INFO - stage1_gradient_single_runtime: 0.002644777297973633
2023-09-28 23:24:55,506 - utils - INFO -  epoch: 2, all client loss: [0.7484148144721985, 0.668014407157898], all pred client disparities: [0.018453404307365417, 0.007147639989852905], all client disparities: [0.0, 0.0013371855020523071], all client accs: [0.2590799033641815, 0.6853925585746765],  alpha_performance: tensor([0.4056, 0.5944], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:55,751 - utils - INFO - stage1_gradient_single_runtime: 0.002306222915649414
2023-09-28 23:24:55,752 - utils - INFO -  epoch: 3, all client loss: [0.7507070899009705, 0.6657418608665466], all pred client disparities: [0.0162152461707592, 0.005475737154483795], all client disparities: [0.0, 0.0031747259199619293], all client accs: [0.2590799033641815, 0.6926091909408569],  alpha_performance: tensor([0.3846, 0.6154], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:55,985 - utils - INFO - stage1_gradient_single_runtime: 0.002292156219482422
2023-09-28 23:24:55,986 - utils - INFO -  epoch: 4, all client loss: [0.7528290748596191, 0.663637101650238], all pred client disparities: [0.014253940433263779, 0.003988325595855713], all client disparities: [0.0, 0.002161804586648941], all client accs: [0.2590799033641815, 0.707882285118103],  alpha_performance: tensor([0.3635, 0.6365], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:56,217 - utils - INFO - stage1_gradient_single_runtime: 0.002335071563720703
2023-09-28 23:24:56,218 - utils - INFO -  epoch: 5, all client loss: [0.7547924518585205, 0.6616883873939514], all pred client disparities: [0.012523598968982697, 0.002660326659679413], all client disparities: [0.0, 0.0025481022894382477], all client accs: [0.2590799033641815, 0.7130458950996399],  alpha_performance: tensor([0.3424, 0.6576], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:56,452 - utils - INFO - stage1_gradient_single_runtime: 0.0022661685943603516
2023-09-28 23:24:56,453 - utils - INFO -  epoch: 6, all client loss: [0.7566085457801819, 0.6598851084709167], all pred client disparities: [0.010986238718032837, 0.0014690607786178589], all client disparities: [0.0, 0.004803319461643696], all client accs: [0.2590799033641815, 0.7164986729621887],  alpha_performance: tensor([0.3213, 0.6787], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:56,686 - utils - INFO - stage1_gradient_single_runtime: 0.0056951045989990234
2023-09-28 23:24:56,687 - utils - INFO -  epoch: 7, all client loss: [0.7582873702049255, 0.658217191696167], all pred client disparities: [0.009610369801521301, 0.0003943219780921936], all client disparities: [0.0, 0.003738267347216606], all client accs: [0.2590799033641815, 0.7257994413375854],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:56,915 - utils - INFO - stage1_gradient_single_runtime: 0.002292156219482422
2023-09-28 23:24:56,916 - utils - INFO -  epoch: 8, all client loss: [0.7530375123023987, 0.6578853726387024], all pred client disparities: [0.011183690279722214, 0.001392081379890442], all client disparities: [0.0, 0.004229141399264336], all client accs: [0.2590799033641815, 0.7274169325828552],  alpha_performance: tensor([0.3267, 0.6733], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:57,146 - utils - INFO - stage1_gradient_single_runtime: 0.0020656585693359375
2023-09-28 23:24:57,147 - utils - INFO -  epoch: 9, all client loss: [0.7547610998153687, 0.656173586845398], all pred client disparities: [0.009658955037593842, 0.00024875253438949585], all client disparities: [0.0, 0.005680385045707226], all client accs: [0.2590799033641815, 0.728878915309906],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:57,379 - utils - INFO - stage1_gradient_single_runtime: 0.00229644775390625
2023-09-28 23:24:57,380 - utils - INFO -  epoch: 10, all client loss: [0.7496073842048645, 0.6558413505554199], all pred client disparities: [0.011255905032157898, 0.0012240633368492126], all client disparities: [0.0, 0.004803424701094627], all client accs: [0.2590799033641815, 0.7306830883026123],  alpha_performance: tensor([0.3304, 0.6696], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:57,608 - utils - INFO - stage1_gradient_single_runtime: 0.002093791961669922
2023-09-28 23:24:57,609 - utils - INFO -  epoch: 11, all client loss: [0.7513700127601624, 0.6540911793708801], all pred client disparities: [0.00957275927066803, 1.2315811545704491e-05], all client disparities: [0.0, 0.0059728436172008514], all client accs: [0.2590799033641815, 0.7342913746833801],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:57,840 - utils - INFO - stage1_gradient_single_runtime: 0.0022819042205810547
2023-09-28 23:24:57,841 - utils - INFO -  epoch: 12, all client loss: [0.7463072538375854, 0.6537598967552185], all pred client disparities: [0.011176064610481262, 0.0009612813591957092], all client disparities: [0.0, 0.005816241726279259], all client accs: [0.2590799033641815, 0.7343224883079529],  alpha_performance: tensor([0.3321, 0.6679], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:58,068 - utils - INFO - stage1_gradient_single_runtime: 0.002139568328857422
2023-09-28 23:24:58,069 - utils - INFO -  epoch: 13, all client loss: [0.7481034398078918, 0.6519767642021179], all pred client disparities: [0.009324226528406143, 0.00031929463148117065], all client disparities: [0.0, 0.006766394712030888], all client accs: [0.2590799033641815, 0.7374953031539917],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:58,306 - utils - INFO - stage1_gradient_single_runtime: 0.0022859573364257812
2023-09-28 23:24:58,307 - utils - INFO -  epoch: 14, all client loss: [0.743127167224884, 0.6516475081443787], all pred client disparities: [0.010914210230112076, 0.0005998760461807251], all client disparities: [0.0, 0.006755968555808067], all client accs: [0.2590799033641815, 0.737433135509491],  alpha_performance: tensor([0.3317, 0.6683], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:58,537 - utils - INFO - stage1_gradient_single_runtime: 0.002087116241455078
2023-09-28 23:24:58,538 - utils - INFO -  epoch: 15, all client loss: [0.7449513077735901, 0.6498368978500366], all pred client disparities: [0.008882246911525726, 0.000750809907913208], all client disparities: [0.0, 0.0060459328815341], all client accs: [0.2590799033641815, 0.7399837970733643],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:58,772 - utils - INFO - stage1_gradient_single_runtime: 0.0022835731506347656
2023-09-28 23:24:58,773 - utils - INFO -  epoch: 16, all client loss: [0.7400572299957275, 0.6495105624198914], all pred client disparities: [0.010436691343784332, 0.00013566017150878906], all client disparities: [0.0, 0.008019434288144112], all client accs: [0.2590799033641815, 0.73870849609375],  alpha_performance: tensor([0.3288, 0.6712], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:59,003 - utils - INFO - stage1_gradient_single_runtime: 0.002084970474243164
2023-09-28 23:24:59,004 - utils - INFO -  epoch: 17, all client loss: [0.7419041395187378, 0.6476778388023376], all pred client disparities: [0.00821133702993393, 0.0012874975800514221], all client disparities: [0.0, 0.005764005705714226], all client accs: [0.2590799033641815, 0.7427833676338196],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:59,228 - utils - INFO - stage1_gradient_single_runtime: 0.002057790756225586
2023-09-28 23:24:59,229 - utils - INFO -  epoch: 18, all client loss: [0.7370885014533997, 0.6473550796508789], all pred client disparities: [0.009705327451229095, 0.0004362538456916809], all client disparities: [0.0, 0.007873255759477615], all client accs: [0.2590799033641815, 0.7421301603317261],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:59,455 - utils - INFO - stage1_gradient_single_runtime: 0.0023016929626464844
2023-09-28 23:24:59,456 - utils - INFO -  epoch: 19, all client loss: [0.7324391007423401, 0.6470034718513489], all pred client disparities: [0.011331089772284031, 0.0004670172929763794], all client disparities: [0.01485507283359766, 0.005294404923915863], all client accs: [0.2615012228488922, 0.7413213849067688],  alpha_performance: tensor([0.3438, 0.6562], device='cuda:0', dtype=torch.float64)
2023-09-28 23:24:59,548 - utils - INFO - valid: True, epoch: 19, loss: [0.7276000380516052, 0.6448180079460144], accuracy: [0.3093922734260559, 0.7516149282455444], mean_accuracy:0.5305036008358002,variance_accuracy:0.22111132740974426, disparity: [0.004545454401522875, 0.00962492823600769], mean_disparity:0.007085191318765283,variance_disparity:0.002539736917242408, pred_disparity: [0.028567451983690262, 0.010718785226345062]
2023-09-28 23:24:59,676 - utils - INFO - global_valid: True, epoch: 19,  global_loss: 0.6457383632659912, global_accuracy: 0.6162403938859605,  global_disparity:0.008835839107632637, global_pred_disparity: 0.008872494101524353,
2023-09-28 23:24:59,905 - utils - INFO - stage1_gradient_single_runtime: 0.002080202102661133
2023-09-28 23:24:59,905 - utils - INFO -  epoch: 20, all client loss: [0.7344647645950317, 0.6449951529502869], all pred client disparities: [0.008461341261863708, 0.0012318789958953857], all client disparities: [0.01485507283359766, 0.009220238775014877], all client accs: [0.2615012228488922, 0.7456762194633484],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:00,135 - utils - INFO - stage1_gradient_single_runtime: 0.0020914077758789062
2023-09-28 23:25:00,135 - utils - INFO -  epoch: 21, all client loss: [0.7298765182495117, 0.6446521282196045], all pred client disparities: [0.009961351752281189, 0.00037248432636260986], all client disparities: [0.01485507283359766, 0.0048663001507520676], all client accs: [0.2615012228488922, 0.7443698048591614],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:00,363 - utils - INFO - stage1_gradient_single_runtime: 0.002279043197631836
2023-09-28 23:25:00,364 - utils - INFO -  epoch: 22, all client loss: [0.7254457473754883, 0.6442807912826538], all pred client disparities: [0.011539846658706665, 0.0005337521433830261], all client disparities: [0.01485507283359766, 0.007831867784261703], all client accs: [0.2615012228488922, 0.7460184097290039],  alpha_performance: tensor([0.3458, 0.6542], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:00,593 - utils - INFO - stage1_gradient_single_runtime: 0.002122163772583008
2023-09-28 23:25:00,594 - utils - INFO -  epoch: 23, all client loss: [0.7276178002357483, 0.6421292424201965], all pred client disparities: [0.007962256669998169, 0.0014514327049255371], all client disparities: [0.01485507283359766, 0.009773772209882736], all client accs: [0.2615012228488922, 0.747573733329773],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:00,827 - utils - INFO - stage1_gradient_single_runtime: 0.0020623207092285156
2023-09-28 23:25:00,828 - utils - INFO -  epoch: 24, all client loss: [0.7232335209846497, 0.6417710781097412], all pred client disparities: [0.009341374039649963, 0.0005947500467300415], all client disparities: [0.005797102116048336, 0.010202091187238693], all client accs: [0.2736077606678009, 0.7499688863754272],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:01,062 - utils - INFO - stage1_gradient_single_runtime: 0.0026721954345703125
2023-09-28 23:25:01,063 - utils - INFO -  epoch: 25, all client loss: [0.718998908996582, 0.6413853764533997], all pred client disparities: [0.010726675391197205, 0.0003046691417694092], all client disparities: [0.007971016690135002, 0.012958716601133347], all client accs: [0.2905569076538086, 0.7481335997581482],  alpha_performance: tensor([0.3316, 0.6684], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:01,307 - utils - INFO - stage1_gradient_single_runtime: 0.0022711753845214844
2023-09-28 23:25:01,308 - utils - INFO -  epoch: 26, all client loss: [0.7212868928909302, 0.6391205787658691], all pred client disparities: [0.006373196840286255, 0.001986168324947357], all client disparities: [0.005797102116048336, 0.011267144232988358], all client accs: [0.2736077606678009, 0.7524573802947998],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:01,550 - utils - INFO - stage1_gradient_single_runtime: 0.0021064281463623047
2023-09-28 23:25:01,551 - utils - INFO -  epoch: 27, all client loss: [0.7170867323875427, 0.6387516856193542], all pred client disparities: [0.007484644651412964, 0.0011395961046218872], all client disparities: [0.024637682363390923, 0.015057437121868134], all client accs: [0.2905569076538086, 0.7517730593681335],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:01,782 - utils - INFO - stage1_gradient_single_runtime: 0.002049684524536133
2023-09-28 23:25:01,783 - utils - INFO -  epoch: 28, all client loss: [0.7130292654037476, 0.6383561491966248], all pred client disparities: [0.008522510528564453, 0.00025326013565063477], all client disparities: [0.03079710528254509, 0.017573941498994827], all client accs: [0.32445523142814636, 0.75083988904953],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:02,017 - utils - INFO - stage1_gradient_single_runtime: 0.0025811195373535156
2023-09-28 23:25:02,018 - utils - INFO -  epoch: 29, all client loss: [0.7091085910797119, 0.6379345655441284], all pred client disparities: [0.009427174925804138, 0.0006653517484664917], all client disparities: [0.02355072647333145, 0.014827955514192581], all client accs: [0.33414044976234436, 0.7509642839431763],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:02,249 - utils - INFO - stage1_gradient_single_runtime: 0.00274658203125
2023-09-28 23:25:02,250 - utils - INFO -  epoch: 30, all client loss: [0.7053191661834717, 0.6374875903129578], all pred client disparities: [0.010140717029571533, 0.0016086548566818237], all client disparities: [0.004347816109657288, 0.006746701896190643], all client accs: [0.4237288236618042, 0.7548525333404541],  alpha_performance: tensor([0.2676, 0.7324], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:02,485 - utils - INFO - stage1_gradient_single_runtime: 0.002081155776977539
2023-09-28 23:25:02,486 - utils - INFO -  epoch: 31, all client loss: [0.7079417705535889, 0.6348970532417297], all pred client disparities: [0.0036643147468566895, 0.0015981942415237427], all client disparities: [0.0365942046046257, 0.017574049532413483], all client accs: [0.33656176924705505, 0.7542615532875061],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:02,712 - utils - INFO - stage1_gradient_single_runtime: 0.002093076705932617
2023-09-28 23:25:02,713 - utils - INFO -  epoch: 32, all client loss: [0.7041559219360352, 0.6344777941703796], all pred client disparities: [0.00400155782699585, 0.0007093697786331177], all client disparities: [0.0061594098806381226, 0.01367969810962677], all client accs: [0.4261501431465149, 0.7576832175254822],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:02,950 - utils - INFO - stage1_gradient_single_runtime: 0.0021266937255859375
2023-09-28 23:25:02,951 - utils - INFO -  epoch: 33, all client loss: [0.7004964351654053, 0.634033739566803], all pred client disparities: [0.004105478525161743, 0.00020429491996765137], all client disparities: [0.00724637508392334, 0.01740754395723343], all client accs: [0.45762714743614197, 0.7593318223953247],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:03,187 - utils - INFO - stage1_gradient_single_runtime: 0.0023581981658935547
2023-09-28 23:25:03,188 - utils - INFO -  epoch: 34, all client loss: [0.6969581246376038, 0.6335656046867371], all pred client disparities: [0.0039411187171936035, 0.001135319471359253], all client disparities: [0.003985509276390076, 0.019861385226249695], all client accs: [0.46731236577033997, 0.7586786150932312],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:03,447 - utils - INFO - stage1_gradient_single_runtime: 0.0022916793823242188
2023-09-28 23:25:03,449 - utils - INFO -  epoch: 35, all client loss: [0.6935359835624695, 0.6330741047859192], all pred client disparities: [0.003485769033432007, 0.0020765066146850586], all client disparities: [0.005072459578514099, 0.010464534163475037], all client accs: [0.47941890358924866, 0.7587718963623047],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:03,680 - utils - INFO - stage1_gradient_single_runtime: 0.002208709716796875
2023-09-28 23:25:03,681 - utils - INFO -  epoch: 36, all client loss: [0.6902253031730652, 0.6325598955154419], all pred client disparities: [0.0027299970388412476, 0.0030210912227630615], all client disparities: [0.08768114447593689, 0.006235919892787933], all client accs: [0.610169529914856, 0.7590829730033875],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:03,921 - utils - INFO - stage1_gradient_single_runtime: 0.0021212100982666016
2023-09-28 23:25:03,921 - utils - INFO -  epoch: 37, all client loss: [0.6870214939117432, 0.6320235729217529], all pred client disparities: [0.001677393913269043, 0.003962904214859009], all client disparities: [0.07789856195449829, 0.005029961466789246], all client accs: [0.6416465044021606, 0.7633134126663208],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:04,159 - utils - INFO - stage1_gradient_single_runtime: 0.002173185348510742
2023-09-28 23:25:04,160 - utils - INFO -  epoch: 38, all client loss: [0.6839203834533691, 0.6314660310745239], all pred client disparities: [0.0003434717655181885, 0.004896268248558044], all client disparities: [0.046376824378967285, 0.0031040608882904053], all client accs: [0.646489143371582, 0.7618825435638428],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:04,415 - utils - INFO - stage1_gradient_single_runtime: 0.002093076705932617
2023-09-28 23:25:04,416 - utils - INFO -  epoch: 39, all client loss: [0.6809176206588745, 0.6308877468109131], all pred client disparities: [0.001246333122253418, 0.005816236138343811], all client disparities: [0.05181160569190979, 0.0035114139318466187], all client accs: [0.6489104628562927, 0.7620380520820618],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:04,525 - utils - INFO - valid: True, epoch: 39, loss: [0.6805871725082397, 0.629885196685791], accuracy: [0.5856353640556335, 0.7665838599205017], mean_accuracy:0.6761096119880676,variance_accuracy:0.09047424793243408, disparity: [0.156060591340065, 0.020266711711883545], mean_disparity:0.08816365152597427,variance_disparity:0.06789693981409073, pred_disparity: [0.0419514924287796, 0.0070696622133255005]
2023-09-28 23:25:04,653 - utils - INFO - global_valid: True, epoch: 39,  global_loss: 0.6304488778114319, global_accuracy: 0.7724294464339893,  global_disparity:0.014532700181007385, global_pred_disparity: 0.00589105486869812,
2023-09-28 23:25:04,922 - utils - INFO - stage1_gradient_single_runtime: 0.0021610260009765625
2023-09-28 23:25:04,922 - utils - INFO -  epoch: 40, all client loss: [0.6780092716217041, 0.6302897334098816], all pred client disparities: [0.003058910369873047, 0.006718561053276062], all client disparities: [0.05543479323387146, 0.002523362636566162], all client accs: [0.6489104628562927, 0.7662062644958496],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:05,195 - utils - INFO - stage1_gradient_single_runtime: 0.0020589828491210938
2023-09-28 23:25:05,196 - utils - INFO -  epoch: 41, all client loss: [0.6751917600631714, 0.6296724081039429], all pred client disparities: [0.0050560832023620605, 0.007599547505378723], all client disparities: [0.0626811683177948, 0.006261199712753296], all client accs: [0.65617436170578, 0.7679793238639832],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:05,448 - utils - INFO - stage1_gradient_single_runtime: 0.0020935535430908203
2023-09-28 23:25:05,449 - utils - INFO -  epoch: 42, all client loss: [0.6724612712860107, 0.6290367245674133], all pred client disparities: [0.007197052240371704, 0.008456185460090637], all client disparities: [0.046014487743377686, 0.005279555916786194], all client accs: [0.6585956811904907, 0.7648998498916626],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:05,703 - utils - INFO - stage1_gradient_single_runtime: 0.0020589828491210938
2023-09-28 23:25:05,704 - utils - INFO -  epoch: 43, all client loss: [0.6698144674301147, 0.6283831596374512], all pred client disparities: [0.009440422058105469, 0.009286165237426758], all client disparities: [0.041666656732559204, 0.015438780188560486], all client accs: [0.7070218324661255, 0.7665795683860779],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:05,970 - utils - INFO - stage1_gradient_single_runtime: 0.002330303192138672
2023-09-28 23:25:05,971 - utils - INFO -  epoch: 44, all client loss: [0.6672479510307312, 0.6277126669883728], all pred client disparities: [0.011746346950531006, 0.010087579488754272], all client disparities: [0.08804348111152649, 0.014321506023406982], all client accs: [0.7191283702850342, 0.7648998498916626],  alpha_performance: tensor([0.4916, 0.5084], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:06,233 - utils - INFO - stage1_gradient_single_runtime: 0.002500295639038086
2023-09-28 23:25:06,234 - utils - INFO -  epoch: 45, all client loss: [0.6681520342826843, 0.6268140077590942], all pred client disparities: [0.00903370976448059, 0.009675264358520508], all client disparities: [0.038043469190597534, 0.015313461422920227], all client accs: [0.7118644118309021, 0.7661129832267761],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:06,488 - utils - INFO - stage1_gradient_single_runtime: 0.0022881031036376953
2023-09-28 23:25:06,490 - utils - INFO -  epoch: 46, all client loss: [0.6656209230422974, 0.6261477470397949], all pred client disparities: [0.011387944221496582, 0.010464698076248169], all client disparities: [0.08260869979858398, 0.014321506023406982], all client accs: [0.7263922691345215, 0.7649931311607361],  alpha_performance: tensor([0.4971, 0.5029], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:06,723 - utils - INFO - stage1_gradient_single_runtime: 0.0022873878479003906
2023-09-28 23:25:06,725 - utils - INFO -  epoch: 47, all client loss: [0.6665490865707397, 0.6252253651618958], all pred client disparities: [0.008722811937332153, 0.010019555687904358], all client disparities: [0.06594201922416687, 0.01538655161857605], all client accs: [0.7239709496498108, 0.7664551138877869],  alpha_performance: tensor([0.4981, 0.5019], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:06,964 - utils - INFO - stage1_gradient_single_runtime: 0.0020949840545654297
2023-09-28 23:25:06,965 - utils - INFO -  epoch: 48, all client loss: [0.6674468517303467, 0.6243330836296082], all pred client disparities: [0.006028681993484497, 0.009612992405891418], all client disparities: [0.038043469190597534, 0.009289011359214783], all client accs: [0.7118644118309021, 0.7679171562194824],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:07,199 - utils - INFO - stage1_gradient_single_runtime: 0.0029556751251220703
2023-09-28 23:25:07,200 - utils - INFO -  epoch: 49, all client loss: [0.6649073958396912, 0.6236923336982727], all pred client disparities: [0.008439391851425171, 0.010408177971839905], all client disparities: [0.08260869979858398, 0.0073259323835372925], all client accs: [0.7263922691345215, 0.7672638893127441],  alpha_performance: tensor([0.5038, 0.4962], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:07,439 - utils - INFO - stage1_gradient_single_runtime: 0.002255678176879883
2023-09-28 23:25:07,440 - utils - INFO -  epoch: 50, all client loss: [0.6658300161361694, 0.6227753758430481], all pred client disparities: [0.005789369344711304, 0.009968608617782593], all client disparities: [0.04746377468109131, 0.010949209332466125], all client accs: [0.7239709496498108, 0.7683215141296387],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:07,617 - utils - INFO - stage1_gradient_single_runtime: 0.0022792816162109375
2023-09-28 23:25:07,618 - utils - INFO -  epoch: 51, all client loss: [0.6633247137069702, 0.6221391558647156], all pred client disparities: [0.008245199918746948, 0.010752320289611816], all client disparities: [0.08079710602760315, 0.007691368460655212], all client accs: [0.7263922691345215, 0.7677615880966187],  alpha_performance: tensor([0.5090, 0.4910], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:07,847 - utils - INFO - stage1_gradient_single_runtime: 0.0022399425506591797
2023-09-28 23:25:07,848 - utils - INFO -  epoch: 52, all client loss: [0.6642688512802124, 0.6212009191513062], all pred client disparities: [0.005637675523757935, 0.010283097624778748], all client disparities: [0.08079710602760315, 0.01188892126083374], all client accs: [0.7288135886192322, 0.7686325907707214],  alpha_performance: tensor([0.5097, 0.4903], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:08,073 - utils - INFO - stage1_gradient_single_runtime: 0.0020895004272460938
2023-09-28 23:25:08,074 - utils - INFO -  epoch: 53, all client loss: [0.6651800870895386, 0.6202952861785889], all pred client disparities: [0.0030067265033721924, 0.009854614734649658], all client disparities: [0.03586956858634949, 0.014342769980430603], all client accs: [0.7118644118309021, 0.7699390053749084],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:08,301 - utils - INFO - stage1_gradient_single_runtime: 0.0022890567779541016
2023-09-28 23:25:08,302 - utils - INFO -  epoch: 54, all client loss: [0.6626647114753723, 0.6196848750114441], all pred client disparities: [0.005513221025466919, 0.010644510388374329], all client disparities: [0.08079710602760315, 0.01149220671504736], all client accs: [0.7263922691345215, 0.7685703635215759],  alpha_performance: tensor([0.5151, 0.4849], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:08,525 - utils - INFO - stage1_gradient_single_runtime: 0.002351045608520508
2023-09-28 23:25:08,525 - utils - INFO -  epoch: 55, all client loss: [0.6635985374450684, 0.6187569499015808], all pred client disparities: [0.002921491861343384, 0.01018589735031128], all client disparities: [0.04746377468109131, 0.01426967978477478], all client accs: [0.7215496897697449, 0.7703123092651367],  alpha_performance: tensor([0.5150, 0.4850], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:08,749 - utils - INFO - stage1_gradient_single_runtime: 0.002062559127807617
2023-09-28 23:25:08,750 - utils - INFO -  epoch: 56, all client loss: [0.6644967794418335, 0.617864191532135], all pred client disparities: [0.000313490629196167, 0.009769275784492493], all client disparities: [0.03586956858634949, 0.01594039797782898], all client accs: [0.7118644118309021, 0.7720853686332703],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:08,977 - utils - INFO - stage1_gradient_single_runtime: 0.0023009777069091797
2023-09-28 23:25:08,977 - utils - INFO -  epoch: 57, all client loss: [0.6619730591773987, 0.6172782778739929], all pred client disparities: [0.0028665661811828613, 0.010564103722572327], all client disparities: [0.07898551225662231, 0.016190946102142334], all client accs: [0.7263922691345215, 0.7712454795837402],  alpha_performance: tensor([0.5207, 0.4793], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:09,201 - utils - INFO - stage1_gradient_single_runtime: 0.0022428035736083984
2023-09-28 23:25:09,202 - utils - INFO -  epoch: 58, all client loss: [0.6628952026367188, 0.6163619756698608], all pred client disparities: [0.0002944767475128174, 0.010116681456565857], all client disparities: [0.03768116235733032, 0.019406959414482117], all client accs: [0.7094431519508362, 0.7723963856697083],  alpha_performance: tensor([0.5200, 0.4800], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:09,429 - utils - INFO - stage1_gradient_single_runtime: 0.0021364688873291016
2023-09-28 23:25:09,430 - utils - INFO -  epoch: 59, all client loss: [0.663779616355896, 0.6154829859733582], all pred client disparities: [0.002287447452545166, 0.00971195101737976], all client disparities: [0.08586955070495605, 0.017172574996948242], all client accs: [0.7021791934967041, 0.7720853686332703],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:09,513 - utils - INFO - valid: True, epoch: 59, loss: [0.6669179201126099, 0.6144903898239136], accuracy: [0.6574586033821106, 0.7740994095802307], mean_accuracy:0.7157790064811707,variance_accuracy:0.05832040309906006, disparity: [0.03939393162727356, 0.00024336576461791992], mean_disparity:0.01981864869594574,variance_disparity:0.01957528293132782, pred_disparity: [0.04937657713890076, 0.00232754647731781]
2023-09-28 23:25:09,641 - utils - INFO - global_valid: True, epoch: 59,  global_loss: 0.6150732636451721, global_accuracy: 0.7888850206199642,  global_disparity:0.00017787516117095947, global_pred_disparity: 0.0015836060047149658,
2023-09-28 23:25:09,863 - utils - INFO - stage1_gradient_single_runtime: 0.002295970916748047
2023-09-28 23:25:09,864 - utils - INFO -  epoch: 60, all client loss: [0.6612492203712463, 0.6149203777313232], all pred client disparities: [0.00030857324600219727, 0.010510683059692383], all client disparities: [0.0043478310108184814, 0.019772395491600037], all client accs: [0.7118644118309021, 0.7723653316497803],  alpha_performance: tensor([0.5259, 0.4741], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:10,087 - utils - INFO - stage1_gradient_single_runtime: 0.002239227294921875
2023-09-28 23:25:10,088 - utils - INFO -  epoch: 61, all client loss: [0.6621586680412292, 0.614016592502594], all pred client disparities: [0.002240777015686035, 0.010074421763420105], all client disparities: [0.03768116235733032, 0.020106568932533264], all client accs: [0.7094431519508362, 0.7730185389518738],  alpha_performance: tensor([0., 1.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:10,316 - utils - INFO - stage1_gradient_single_runtime: 0.0020797252655029297
2023-09-28 23:25:10,317 - utils - INFO -  epoch: 62, all client loss: [0.661971390247345, 0.6124032139778137], all pred client disparities: [0.0074555277824401855, 0.006831347942352295], all client disparities: [0.07898551225662231, 0.012244254350662231], all client accs: [0.7239709496498108, 0.7739517092704773],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:10,543 - utils - INFO - stage1_gradient_single_runtime: 0.002074003219604492
2023-09-28 23:25:10,543 - utils - INFO -  epoch: 63, all client loss: [0.6594681143760681, 0.6118637919425964], all pred client disparities: [0.009898185729980469, 0.007624760270118713], all client disparities: [0.07898551225662231, 0.011179104447364807], all client accs: [0.7239709496498108, 0.7730496525764465],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:10,778 - utils - INFO - stage1_gradient_single_runtime: 0.002323627471923828
2023-09-28 23:25:10,779 - utils - INFO -  epoch: 64, all client loss: [0.6570421457290649, 0.6113077402114868], all pred client disparities: [0.012332171201705933, 0.008394330739974976], all client disparities: [0.07717391848564148, 0.011210381984710693], all client accs: [0.7263922691345215, 0.7726763486862183],  alpha_performance: tensor([0.5173, 0.4827], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:11,014 - utils - INFO - stage1_gradient_single_runtime: 0.0025627613067626953
2023-09-28 23:25:11,015 - utils - INFO -  epoch: 65, all client loss: [0.6579527854919434, 0.6104029417037964], all pred client disparities: [0.01000872254371643, 0.007909640669822693], all client disparities: [0.07717391848564148, 0.010604813694953918], all client accs: [0.7263922691345215, 0.7730807662010193],  alpha_performance: tensor([0.5199, 0.4801], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:11,261 - utils - INFO - stage1_gradient_single_runtime: 0.0020961761474609375
2023-09-28 23:25:11,262 - utils - INFO -  epoch: 66, all client loss: [0.6588367819786072, 0.6095244288444519], all pred client disparities: [0.007644623517990112, 0.007466912269592285], all client disparities: [0.07717391848564148, 0.011816158890724182], all client accs: [0.7263922691345215, 0.7727385759353638],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:11,488 - utils - INFO - stage1_gradient_single_runtime: 0.0022759437561035156
2023-09-28 23:25:11,489 - utils - INFO -  epoch: 67, all client loss: [0.656402051448822, 0.6089918613433838], all pred client disparities: [0.010134369134902954, 0.008238822221755981], all client disparities: [0.07717391848564148, 0.010030537843704224], all client accs: [0.7263922691345215, 0.77314293384552],  alpha_performance: tensor([0.5234, 0.4766], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:11,716 - utils - INFO - stage1_gradient_single_runtime: 0.002073526382446289
2023-09-28 23:25:11,716 - utils - INFO -  epoch: 68, all client loss: [0.6573001742362976, 0.6080994009971619], all pred client disparities: [0.007813781499862671, 0.0077724456787109375], all client disparities: [0.07717391848564148, 0.015251323580741882], all client accs: [0.7263922691345215, 0.773547351360321],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:11,942 - utils - INFO - stage1_gradient_single_runtime: 0.0022847652435302734
2023-09-28 23:25:11,943 - utils - INFO -  epoch: 69, all client loss: [0.6548988819122314, 0.607570230960846], all pred client disparities: [0.010318875312805176, 0.008533582091331482], all client disparities: [0.07717391848564148, 0.009936600923538208], all client accs: [0.7263922691345215, 0.77314293384552],  alpha_performance: tensor([0.5264, 0.4736], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:12,169 - utils - INFO - stage1_gradient_single_runtime: 0.0020880699157714844
2023-09-28 23:25:12,170 - utils - INFO -  epoch: 70, all client loss: [0.6558084487915039, 0.6066664457321167], all pred client disparities: [0.008040964603424072, 0.008046478033065796], all client disparities: [0.07717391848564148, 0.013956576585769653], all client accs: [0.7263922691345215, 0.7737339735031128],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:12,394 - utils - INFO - stage1_gradient_single_runtime: 0.002258777618408203
2023-09-28 23:25:12,395 - utils - INFO -  epoch: 71, all client loss: [0.653438925743103, 0.6061411499977112], all pred client disparities: [0.010555386543273926, 0.00879719853401184], all client disparities: [0.07717391848564148, 0.013319626450538635], all client accs: [0.7263922691345215, 0.7734851241111755],  alpha_performance: tensor([0.5290, 0.4710], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:12,619 - utils - INFO - stage1_gradient_single_runtime: 0.0020780563354492188
2023-09-28 23:25:12,620 - utils - INFO -  epoch: 72, all client loss: [0.6543574929237366, 0.6052286624908447], all pred client disparities: [0.008319616317749023, 0.008291870355606079], all client disparities: [0.07717391848564148, 0.014029666781425476], all client accs: [0.7263922691345215, 0.7738894820213318],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:12,845 - utils - INFO - stage1_gradient_single_runtime: 0.002287626266479492
2023-09-28 23:25:12,846 - utils - INFO -  epoch: 73, all client loss: [0.6520184278488159, 0.6047077178955078], all pred client disparities: [0.010837793350219727, 0.009032532572746277], all client disparities: [0.07536232471466064, 0.013319626450538635], all client accs: [0.7288135886192322, 0.7735784649848938],  alpha_performance: tensor([0.5311, 0.4689], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:13,074 - utils - INFO - stage1_gradient_single_runtime: 0.002063751220703125
2023-09-28 23:25:13,075 - utils - INFO -  epoch: 74, all client loss: [0.6529435515403748, 0.6037887334823608], all pred client disparities: [0.008644044399261475, 0.008511275053024292], all client disparities: [0.07717391848564148, 0.013674646615982056], all client accs: [0.7263922691345215, 0.7740450501441956],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:13,299 - utils - INFO - stage1_gradient_single_runtime: 0.0022759437561035156
2023-09-28 23:25:13,300 - utils - INFO -  epoch: 75, all client loss: [0.6506337523460388, 0.603272557258606], all pred client disparities: [0.011160701513290405, 0.009242162108421326], all client disparities: [0.07355073094367981, 0.01274535059928894], all client accs: [0.7312349081039429, 0.7738584280014038],  alpha_performance: tensor([0.5328, 0.4672], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:13,527 - utils - INFO - stage1_gradient_single_runtime: 0.0020711421966552734
2023-09-28 23:25:13,527 - utils - INFO -  epoch: 76, all client loss: [0.6515634059906006, 0.602349042892456], all pred client disparities: [0.009008973836898804, 0.008707135915756226], all client disparities: [0.07355073094367981, 0.013747736811637878], all client accs: [0.7312349081039429, 0.7740761637687683],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:13,750 - utils - INFO - stage1_gradient_single_runtime: 0.002223491668701172
2023-09-28 23:25:13,751 - utils - INFO -  epoch: 77, all client loss: [0.6492817997932434, 0.6018378138542175], all pred client disparities: [0.01151961088180542, 0.009428560733795166], all client disparities: [0.07355073094367981, 0.012755781412124634], all client accs: [0.7288135886192322, 0.7740450501441956],  alpha_performance: tensor([0.5342, 0.4658], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:13,973 - utils - INFO - stage1_gradient_single_runtime: 0.0020952224731445312
2023-09-28 23:25:13,974 - utils - INFO -  epoch: 78, all client loss: [0.6502141356468201, 0.6009117364883423], all pred client disparities: [0.009409904479980469, 0.008881479501724243], all client disparities: [0.07355073094367981, 0.015115559101104736], all client accs: [0.7312349081039429, 0.7741694450378418],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:14,198 - utils - INFO - stage1_gradient_single_runtime: 0.00225830078125
2023-09-28 23:25:14,199 - utils - INFO -  epoch: 79, all client loss: [0.6479596495628357, 0.600405752658844], all pred client disparities: [0.011910110712051392, 0.009593799710273743], all client disparities: [0.07355073094367981, 0.012327656149864197], all client accs: [0.7288135886192322, 0.773547351360321],  alpha_performance: tensor([0.5352, 0.4648], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:14,285 - utils - INFO - valid: True, epoch: 79, loss: [0.6569204926490784, 0.5989381670951843], accuracy: [0.6795580387115479, 0.7752174139022827], mean_accuracy:0.7273877263069153,variance_accuracy:0.04782968759536743, disparity: [0.018181830644607544, 0.005207881331443787], mean_disparity:0.011694855988025665,variance_disparity:0.006486974656581879, pred_disparity: [0.03793337941169739, 0.0023376643657684326]
2023-09-28 23:25:14,413 - utils - INFO - global_valid: True, epoch: 79,  global_loss: 0.5995827913284302, global_accuracy: 0.7959748152692494,  global_disparity:0.0020298808813095093, global_pred_disparity: 0.002448156476020813,
2023-09-28 23:25:14,634 - utils - INFO - stage1_gradient_single_runtime: 0.002147197723388672
2023-09-28 23:25:14,635 - utils - INFO -  epoch: 80, all client loss: [0.6488929986953735, 0.5994786620140076], all pred client disparities: [0.009842783212661743, 0.009036421775817871], all client disparities: [0.07355073094367981, 0.014906734228134155], all client accs: [0.7288135886192322, 0.7748538255691528],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:14,860 - utils - INFO - stage1_gradient_single_runtime: 0.0022644996643066406
2023-09-28 23:25:14,861 - utils - INFO -  epoch: 81, all client loss: [0.6466649770736694, 0.5989781618118286], all pred client disparities: [0.012328654527664185, 0.009739875793457031], all client disparities: [0.07173913717269897, 0.013204723596572876], all client accs: [0.7263922691345215, 0.7741072773933411],  alpha_performance: tensor([0.5360, 0.4640], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:15,086 - utils - INFO - stage1_gradient_single_runtime: 0.0022535324096679688
2023-09-28 23:25:15,087 - utils - INFO -  epoch: 82, all client loss: [0.6475977301597595, 0.5980516076087952], all pred client disparities: [0.010304033756256104, 0.009173601865768433], all client disparities: [0.07173913717269897, 0.014186277985572815], all client accs: [0.7263922691345215, 0.7751026749610901],  alpha_performance: tensor([0.5406, 0.4594], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:15,310 - utils - INFO - stage1_gradient_single_runtime: 0.0021026134490966797
2023-09-28 23:25:15,311 - utils - INFO -  epoch: 83, all client loss: [0.6485086679458618, 0.59714674949646], all pred client disparities: [0.008232325315475464, 0.00864650309085846], all client disparities: [0.07173913717269897, 0.01481279730796814], all client accs: [0.7263922691345215, 0.7760047316551208],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:15,538 - utils - INFO - stage1_gradient_single_runtime: 0.0022611618041992188
2023-09-28 23:25:15,539 - utils - INFO -  epoch: 84, all client loss: [0.646264374256134, 0.5966714024543762], all pred client disparities: [0.010767757892608643, 0.00935623049736023], all client disparities: [0.07173913717269897, 0.013747736811637878], all client accs: [0.7263922691345215, 0.7750093340873718],  alpha_performance: tensor([0.5415, 0.4585], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:15,763 - utils - INFO - stage1_gradient_single_runtime: 0.0020520687103271484
2023-09-28 23:25:15,764 - utils - INFO -  epoch: 85, all client loss: [0.6471763253211975, 0.5957655906677246], all pred client disparities: [0.008737385272979736, 0.008818462491035461], all client disparities: [0.07173913717269897, 0.014885887503623962], all client accs: [0.7263922691345215, 0.7760669589042664],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:15,988 - utils - INFO - stage1_gradient_single_runtime: 0.002252340316772461
2023-09-28 23:25:15,989 - utils - INFO -  epoch: 86, all client loss: [0.6449593901634216, 0.5952947735786438], all pred client disparities: [0.011255621910095215, 0.009519204497337341], all client disparities: [0.07173913717269897, 0.014990255236625671], all client accs: [0.7263922691345215, 0.7753204107284546],  alpha_performance: tensor([0.5421, 0.4579], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:16,214 - utils - INFO - stage1_gradient_single_runtime: 0.002155780792236328
2023-09-28 23:25:16,215 - utils - INFO -  epoch: 87, all client loss: [0.6458709239959717, 0.5943893790245056], all pred client disparities: [0.00926712155342102, 0.00897236168384552], all client disparities: [0.07173913717269897, 0.015032052993774414], all client accs: [0.7263922691345215, 0.7763779759407043],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:16,439 - utils - INFO - stage1_gradient_single_runtime: 0.002240419387817383
2023-09-28 23:25:16,440 - utils - INFO -  epoch: 88, all client loss: [0.6436804533004761, 0.5939233899116516], all pred client disparities: [0.011764466762542725, 0.009664282202720642], all client disparities: [0.06449276208877563, 0.014990255236625671], all client accs: [0.7312349081039429, 0.7753825783729553],  alpha_performance: tensor([0.5424, 0.4576], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:16,665 - utils - INFO - stage1_gradient_single_runtime: 0.0020813941955566406
2023-09-28 23:25:16,666 - utils - INFO -  epoch: 89, all client loss: [0.6445902585983276, 0.5930199027061462], all pred client disparities: [0.00981837511062622, 0.009109675884246826], all client disparities: [0.07173913717269897, 0.015387073159217834], all client accs: [0.7263922691345215, 0.7761291265487671],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:16,890 - utils - INFO - stage1_gradient_single_runtime: 0.0022783279418945312
2023-09-28 23:25:16,890 - utils - INFO -  epoch: 90, all client loss: [0.6424257159233093, 0.592558741569519], all pred client disparities: [0.012291431427001953, 0.009793087840080261], all client disparities: [0.06449276208877563, 0.013549327850341797], all client accs: [0.7312349081039429, 0.7754759192466736],  alpha_performance: tensor([0.5424, 0.4576], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:17,118 - utils - INFO - stage1_gradient_single_runtime: 0.002245187759399414
2023-09-28 23:25:17,119 - utils - INFO -  epoch: 91, all client loss: [0.6433322429656982, 0.591658353805542], all pred client disparities: [0.010388284921646118, 0.009231865406036377], all client disparities: [0.07173913717269897, 0.015387073159217834], all client accs: [0.7263922691345215, 0.7760358452796936],  alpha_performance: tensor([0.5474, 0.4526], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:17,342 - utils - INFO - stage1_gradient_single_runtime: 0.0020904541015625
2023-09-28 23:25:17,343 - utils - INFO -  epoch: 92, all client loss: [0.6442188024520874, 0.5907777547836304], all pred client disparities: [0.00843784213066101, 0.008707493543624878], all client disparities: [0.07173913717269897, 0.006272047758102417], all client accs: [0.7263922691345215, 0.7725830674171448],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:17,565 - utils - INFO - stage1_gradient_single_runtime: 0.0023012161254882812
2023-09-28 23:25:17,566 - utils - INFO -  epoch: 93, all client loss: [0.6420387625694275, 0.5903403162956238], all pred client disparities: [0.010956913232803345, 0.009396746754646301], all client disparities: [0.0626811683177948, 0.01553325355052948], all client accs: [0.7263922691345215, 0.7760669589042664],  alpha_performance: tensor([0.5475, 0.4525], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:17,787 - utils - INFO - stage1_gradient_single_runtime: 0.0020771026611328125
2023-09-28 23:25:17,788 - utils - INFO -  epoch: 94, all client loss: [0.6429238319396973, 0.5894612669944763], all pred client disparities: [0.009048402309417725, 0.008863985538482666], all client disparities: [0.06992754340171814, 0.007441461086273193], all client accs: [0.7215496897697449, 0.7723342180252075],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:18,011 - utils - INFO - stage1_gradient_single_runtime: 0.0022580623626708984
2023-09-28 23:25:18,011 - utils - INFO -  epoch: 95, all client loss: [0.6407707333564758, 0.5890278220176697], all pred client disparities: [0.011540383100509644, 0.009544551372528076], all client disparities: [0.0626811683177948, 0.01596134901046753], all client accs: [0.7263922691345215, 0.7769379019737244],  alpha_performance: tensor([0.5474, 0.4526], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:18,232 - utils - INFO - stage1_gradient_single_runtime: 0.0020673274993896484
2023-09-28 23:25:18,233 - utils - INFO -  epoch: 96, all client loss: [0.6416530013084412, 0.5881516337394714], all pred client disparities: [0.009674400091171265, 0.009004577994346619], all client disparities: [0.0626811683177948, 0.006794095039367676], all client accs: [0.7263922691345215, 0.7724897265434265],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:18,456 - utils - INFO - stage1_gradient_single_runtime: 0.002256631851196289
2023-09-28 23:25:18,457 - utils - INFO -  epoch: 97, all client loss: [0.6395262479782104, 0.5877221822738647], all pred client disparities: [0.01213604211807251, 0.00967666506767273], all client disparities: [0.0626811683177948, 0.015313982963562012], all client accs: [0.7263922691345215, 0.7770001292228699],  alpha_performance: tensor([0.5470, 0.4530], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:18,681 - utils - INFO - stage1_gradient_single_runtime: 0.0022661685943603516
2023-09-28 23:25:18,682 - utils - INFO -  epoch: 98, all client loss: [0.6404045224189758, 0.5868499875068665], all pred client disparities: [0.010313302278518677, 0.009130552411079407], all client disparities: [0.0626811683177948, 0.005645528435707092], all client accs: [0.7263922691345215, 0.772551953792572],  alpha_performance: tensor([0.5521, 0.4479], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:18,912 - utils - INFO - stage1_gradient_single_runtime: 0.0020596981048583984
2023-09-28 23:25:18,912 - utils - INFO -  epoch: 99, all client loss: [0.641264021396637, 0.5859962105751038], all pred client disparities: [0.008443981409072876, 0.008619457483291626], all client disparities: [0.0626811683177948, 0.007900968194007874], all client accs: [0.7263922691345215, 0.7726763486862183],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:18,997 - utils - INFO - valid: True, epoch: 99, loss: [0.6490697264671326, 0.5849934220314026], accuracy: [0.6850829124450684, 0.7753416299819946], mean_accuracy:0.7302122712135315,variance_accuracy:0.045129358768463135, disparity: [0.004545450210571289, 0.003732919692993164], mean_disparity:0.0041391849517822266,variance_disparity:0.0004062652587890625, pred_disparity: [0.03524148464202881, 0.0009207725524902344]
2023-09-28 23:25:19,125 - utils - INFO - global_valid: True, epoch: 99,  global_loss: 0.5857058167457581, global_accuracy: 0.8001374385494118,  global_disparity:0.005990639328956604, global_pred_disparity: 0.0014807283878326416,
2023-09-28 23:25:19,348 - utils - INFO - stage1_gradient_single_runtime: 0.0022716522216796875
2023-09-28 23:25:19,349 - utils - INFO -  epoch: 100, all client loss: [0.6391235589981079, 0.5855889916419983], all pred client disparities: [0.010948359966278076, 0.009296536445617676], all client disparities: [0.0626811683177948, 0.005499348044395447], all client accs: [0.7263922691345215, 0.7726141810417175],  alpha_performance: tensor([0.5518, 0.4482], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:19,572 - utils - INFO - stage1_gradient_single_runtime: 0.0020706653594970703
2023-09-28 23:25:19,573 - utils - INFO -  epoch: 101, all client loss: [0.6399806141853333, 0.5847377777099609], all pred client disparities: [0.009121298789978027, 0.008777573704719543], all client disparities: [0.0626811683177948, 0.00819331407546997], all client accs: [0.7263922691345215, 0.7727385759353638],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:19,800 - utils - INFO - stage1_gradient_single_runtime: 0.002265453338623047
2023-09-28 23:25:19,801 - utils - INFO -  epoch: 102, all client loss: [0.6378673911094666, 0.5843339562416077], all pred client disparities: [0.011592596769332886, 0.00944603979587555], all client disparities: [0.0626811683177948, 0.00514434278011322], all client accs: [0.7263922691345215, 0.7727385759353638],  alpha_performance: tensor([0.5512, 0.4488], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:20,026 - utils - INFO - stage1_gradient_single_runtime: 0.0020732879638671875
2023-09-28 23:25:20,027 - utils - INFO -  epoch: 103, all client loss: [0.6387208104133606, 0.5834861993789673], all pred client disparities: [0.009808510541915894, 0.008920341730117798], all client disparities: [0.0626811683177948, 0.007545933127403259], all client accs: [0.7263922691345215, 0.7728008031845093],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:20,250 - utils - INFO - stage1_gradient_single_runtime: 0.002251148223876953
2023-09-28 23:25:20,251 - utils - INFO -  epoch: 104, all client loss: [0.6366342306137085, 0.5830857753753662], all pred client disparities: [0.012243658304214478, 0.00958038866519928], all client disparities: [0.0626811683177948, 0.012160733342170715], all client accs: [0.7263922691345215, 0.7774978280067444],  alpha_performance: tensor([0.5504, 0.4496], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:20,477 - utils - INFO - stage1_gradient_single_runtime: 0.002275705337524414
2023-09-28 23:25:20,478 - utils - INFO -  epoch: 105, all client loss: [0.6374831199645996, 0.5822426676750183], all pred client disparities: [0.010503411293029785, 0.00904892385005951], all client disparities: [0.0626811683177948, 0.006543546915054321], all client accs: [0.7263922691345215, 0.772831916809082],  alpha_performance: tensor([0.5557, 0.4443], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:20,708 - utils - INFO - stage1_gradient_single_runtime: 0.002072572708129883
2023-09-28 23:25:20,709 - utils - INFO -  epoch: 106, all client loss: [0.6383145451545715, 0.5814167857170105], all pred client disparities: [0.008716940879821777, 0.008550450205802917], all client disparities: [0.0626811683177948, 0.008475244045257568], all client accs: [0.7263922691345215, 0.772831916809082],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:20,941 - utils - INFO - stage1_gradient_single_runtime: 0.0022809505462646484
2023-09-28 23:25:20,941 - utils - INFO -  epoch: 107, all client loss: [0.6362157464027405, 0.5810369253158569], all pred client disparities: [0.01119154691696167, 0.00921475887298584], all client disparities: [0.0626811683177948, 0.0067628175020217896], all client accs: [0.7263922691345215, 0.7727696895599365],  alpha_performance: tensor([0.5549, 0.4451], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:21,171 - utils - INFO - stage1_gradient_single_runtime: 0.002068758010864258
2023-09-28 23:25:21,171 - utils - INFO -  epoch: 108, all client loss: [0.6370439529418945, 0.5802143812179565], all pred client disparities: [0.009448021650314331, 0.008708938956260681], all client disparities: [0.0626811683177948, 0.008412584662437439], all client accs: [0.7263922691345215, 0.7730496525764465],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:21,400 - utils - INFO - stage1_gradient_single_runtime: 0.0023233890533447266
2023-09-28 23:25:21,401 - utils - INFO -  epoch: 109, all client loss: [0.6349727511405945, 0.5798373222351074], all pred client disparities: [0.01188361644744873, 0.00936475396156311], all client disparities: [0.0626811683177948, 0.004820704460144043], all client accs: [0.7263922691345215, 0.7727385759353638],  alpha_performance: tensor([0.5539, 0.4461], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:21,642 - utils - INFO - stage1_gradient_single_runtime: 0.0022797584533691406
2023-09-28 23:25:21,643 - utils - INFO -  epoch: 110, all client loss: [0.6357966661453247, 0.5790188908576965], all pred client disparities: [0.01018381118774414, 0.008852541446685791], all client disparities: [0.0626811683177948, 0.0067628175020217896], all client accs: [0.7263922691345215, 0.772987425327301],  alpha_performance: tensor([0.5591, 0.4409], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:21,816 - utils - INFO - stage1_gradient_single_runtime: 0.0021047592163085938
2023-09-28 23:25:21,816 - utils - INFO -  epoch: 111, all client loss: [0.6366037726402283, 0.5782171487808228], all pred client disparities: [0.008439332246780396, 0.008371978998184204], all client disparities: [0.0626811683177948, 0.010657578706741333], all client accs: [0.7263922691345215, 0.7736717462539673],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:22,175 - utils - INFO - stage1_gradient_single_runtime: 0.002862691879272461
2023-09-28 23:25:22,176 - utils - INFO -  epoch: 112, all client loss: [0.6345223188400269, 0.5778592228889465], all pred client disparities: [0.010911315679550171, 0.009031057357788086], all client disparities: [0.05905798077583313, 0.0067628175020217896], all client accs: [0.7312349081039429, 0.772987425327301],  alpha_performance: tensor([0.5582, 0.4418], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:22,411 - utils - INFO - stage1_gradient_single_runtime: 0.002110719680786133
2023-09-28 23:25:22,412 - utils - INFO -  epoch: 113, all client loss: [0.6353265047073364, 0.5770604610443115], all pred client disparities: [0.009209781885147095, 0.008542656898498535], all client disparities: [0.05905798077583313, 0.011252716183662415], all client accs: [0.7312349081039429, 0.7736717462539673],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:22,639 - utils - INFO - stage1_gradient_single_runtime: 0.0026390552520751953
2023-09-28 23:25:22,640 - utils - INFO -  epoch: 114, all client loss: [0.6332733035087585, 0.5767045617103577], all pred client disparities: [0.011639803647994995, 0.009193256497383118], all client disparities: [0.057246387004852295, 0.006835907697677612], all client accs: [0.7336562275886536, 0.7730185389518738],  alpha_performance: tensor([0.5570, 0.4430], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:22,873 - utils - INFO - stage1_gradient_single_runtime: 0.002435922622680664
2023-09-28 23:25:22,874 - utils - INFO -  epoch: 115, all client loss: [0.6340734362602234, 0.5759098529815674], all pred client disparities: [0.009981989860534668, 0.00869797170162201], all client disparities: [0.057246387004852295, 0.01067844033241272], all client accs: [0.7336562275886536, 0.7735162377357483],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:23,121 - utils - INFO - stage1_gradient_single_runtime: 0.0023119449615478516
2023-09-28 23:25:23,122 - utils - INFO -  epoch: 116, all client loss: [0.6320480108261108, 0.5755560398101807], all pred client disparities: [0.012367576360702515, 0.009340286254882812], all client disparities: [0.057246387004852295, 0.006835907697677612], all client accs: [0.7336562275886536, 0.7730185389518738],  alpha_performance: tensor([0.5555, 0.4445], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:23,347 - utils - INFO - stage1_gradient_single_runtime: 0.002260446548461914
2023-09-28 23:25:23,348 - utils - INFO -  epoch: 117, all client loss: [0.632843017578125, 0.5747663974761963], all pred client disparities: [0.010754287242889404, 0.008839026093482971], all client disparities: [0.057246387004852295, 0.01067844033241272], all client accs: [0.7336562275886536, 0.7734540104866028],  alpha_performance: tensor([0.5609, 0.4391], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:23,576 - utils - INFO - stage1_gradient_single_runtime: 0.002192974090576172
2023-09-28 23:25:23,577 - utils - INFO -  epoch: 118, all client loss: [0.6336228251457214, 0.5739918947219849], all pred client disparities: [0.009096324443817139, 0.008367642760276794], all client disparities: [0.057246387004852295, 0.010657578706741333], all client accs: [0.7336562275886536, 0.7737339735031128],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:23,810 - utils - INFO - stage1_gradient_single_runtime: 0.00228118896484375
2023-09-28 23:25:23,811 - utils - INFO -  epoch: 119, all client loss: [0.6315883994102478, 0.573655903339386], all pred client disparities: [0.011514872312545776, 0.009012669324874878], all client disparities: [0.057246387004852295, 0.010970786213874817], all client accs: [0.7336562275886536, 0.7733607292175293],  alpha_performance: tensor([0.5595, 0.4405], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:23,896 - utils - INFO - valid: True, epoch: 119, loss: [0.6436783671379089, 0.5722359418869019], accuracy: [0.6906077861785889, 0.7767080664634705], mean_accuracy:0.7336579263210297,variance_accuracy:0.043050140142440796, disparity: [0.013636350631713867, 0.009515166282653809], mean_disparity:0.011575758457183838,variance_disparity:0.0020605921745300293, pred_disparity: [0.03522351384162903, 0.000648990273475647]
2023-09-28 23:25:24,021 - utils - INFO - global_valid: True, epoch: 119,  global_loss: 0.5730302333831787, global_accuracy: 0.8014046311751948,  global_disparity:0.011319667100906372, global_pred_disparity: 0.0014482289552688599,
2023-09-28 23:25:24,244 - utils - INFO - stage1_gradient_single_runtime: 0.002117633819580078
2023-09-28 23:25:24,245 - utils - INFO -  epoch: 120, all client loss: [0.6323642730712891, 0.572885274887085], all pred client disparities: [0.009900808334350586, 0.008533984422683716], all client disparities: [0.057246387004852295, 0.010803744196891785], all client accs: [0.7336562275886536, 0.7735784649848938],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:24,484 - utils - INFO - stage1_gradient_single_runtime: 0.0022513866424560547
2023-09-28 23:25:24,485 - utils - INFO -  epoch: 121, all client loss: [0.6303583979606628, 0.5725507736206055], all pred client disparities: [0.012272000312805176, 0.009170696139335632], all client disparities: [0.057246387004852295, 0.011116966605186462], all client accs: [0.7336562275886536, 0.7733917832374573],  alpha_performance: tensor([0.5579, 0.4421], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:24,709 - utils - INFO - stage1_gradient_single_runtime: 0.0026569366455078125
2023-09-28 23:25:24,710 - utils - INFO -  epoch: 122, all client loss: [0.6311295032501221, 0.5717849135398865], all pred client disparities: [0.010702580213546753, 0.008685588836669922], all client disparities: [0.057246387004852295, 0.012046262621879578], all client accs: [0.7336562275886536, 0.7736095786094666],  alpha_performance: tensor([0.5633, 0.4367], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:24,941 - utils - INFO - stage1_gradient_single_runtime: 0.0022592544555664062
2023-09-28 23:25:24,942 - utils - INFO -  epoch: 123, all client loss: [0.6318860054016113, 0.5710335373878479], all pred client disparities: [0.009089499711990356, 0.008229032158851624], all client disparities: [0.057246387004852295, 0.011659964919090271], all client accs: [0.7336562275886536, 0.774418294429779],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:25,192 - utils - INFO - stage1_gradient_single_runtime: 0.0023040771484375
2023-09-28 23:25:25,193 - utils - INFO -  epoch: 124, all client loss: [0.6298727989196777, 0.5707156658172607], all pred client disparities: [0.011490792036056519, 0.00886775553226471], all client disparities: [0.057246387004852295, 0.011973172426223755], all client accs: [0.7336562275886536, 0.774138331413269],  alpha_performance: tensor([0.5617, 0.4383], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:25,417 - utils - INFO - stage1_gradient_single_runtime: 0.0020744800567626953
2023-09-28 23:25:25,417 - utils - INFO -  epoch: 125, all client loss: [0.6306256055831909, 0.5699679255485535], all pred client disparities: [0.009921878576278687, 0.008403509855270386], all client disparities: [0.057246387004852295, 0.011806145310401917], all client accs: [0.7336562275886536, 0.7742627859115601],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:25,642 - utils - INFO - stage1_gradient_single_runtime: 0.002272367477416992
2023-09-28 23:25:25,643 - utils - INFO -  epoch: 126, all client loss: [0.6286414265632629, 0.5696511268615723], all pred client disparities: [0.012272864580154419, 0.009034007787704468], all client disparities: [0.057246387004852295, 0.012046262621879578], all client accs: [0.7336562275886536, 0.7742627859115601],  alpha_performance: tensor([0.5598, 0.4402], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:25,866 - utils - INFO - stage1_gradient_single_runtime: 0.0022439956665039062
2023-09-28 23:25:25,867 - utils - INFO -  epoch: 127, all client loss: [0.6293895244598389, 0.5689079761505127], all pred client disparities: [0.010748714208602905, 0.008563011884689331], all client disparities: [0.057246387004852295, 0.011231869459152222], all client accs: [0.7336562275886536, 0.7743250131607056],  alpha_performance: tensor([0.5652, 0.4348], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:26,091 - utils - INFO - stage1_gradient_single_runtime: 0.002073049545288086
2023-09-28 23:25:26,092 - utils - INFO -  epoch: 128, all client loss: [0.6301237940788269, 0.568178653717041], all pred client disparities: [0.009181857109069824, 0.008119389414787292], all client disparities: [0.057246387004852295, 0.013247072696685791], all client accs: [0.7336562275886536, 0.7743561267852783],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:26,317 - utils - INFO - stage1_gradient_single_runtime: 0.0022497177124023438
2023-09-28 23:25:26,318 - utils - INFO -  epoch: 129, all client loss: [0.6281339526176453, 0.5678772330284119], all pred client disparities: [0.011560052633285522, 0.008751258254051208], all client disparities: [0.057246387004852295, 0.01247437298297882], all client accs: [0.7336562275886536, 0.7742005586624146],  alpha_performance: tensor([0.5633, 0.4367], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:26,542 - utils - INFO - stage1_gradient_single_runtime: 0.0022640228271484375
2023-09-28 23:25:26,542 - utils - INFO -  epoch: 130, all client loss: [0.6288644671440125, 0.5671515464782715], all pred client disparities: [0.01003757119178772, 0.008299753069877625], all client disparities: [0.057246387004852295, 0.012745872139930725], all client accs: [0.7336562275886536, 0.774418294429779],  alpha_performance: tensor([0.5685, 0.4315], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:26,765 - utils - INFO - stage1_gradient_single_runtime: 0.002125978469848633
2023-09-28 23:25:26,766 - utils - INFO -  epoch: 131, all client loss: [0.6295813322067261, 0.5664393901824951], all pred client disparities: [0.008473962545394897, 0.00787489116191864], all client disparities: [0.057246387004852295, 0.011628687381744385], all client accs: [0.7336562275886536, 0.7768756747245789],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:26,991 - utils - INFO - stage1_gradient_single_runtime: 0.0022280216217041016
2023-09-28 23:25:26,992 - utils - INFO -  epoch: 132, all client loss: [0.627587616443634, 0.5661523938179016], all pred client disparities: [0.010877162218093872, 0.008507221937179565], all client disparities: [0.057246387004852295, 0.012745872139930725], all client accs: [0.7336562275886536, 0.7743250131607056],  alpha_performance: tensor([0.5667, 0.4333], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:27,214 - utils - INFO - stage1_gradient_single_runtime: 0.0020499229431152344
2023-09-28 23:25:27,215 - utils - INFO -  epoch: 133, all client loss: [0.6283016800880432, 0.5654430389404297], all pred client disparities: [0.009357541799545288, 0.008073478937149048], all client disparities: [0.057246387004852295, 0.011921048164367676], all client accs: [0.7336562275886536, 0.7768445611000061],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:27,437 - utils - INFO - stage1_gradient_single_runtime: 0.0022706985473632812
2023-09-28 23:25:27,438 - utils - INFO -  epoch: 134, all client loss: [0.626338541507721, 0.5651560425758362], all pred client disparities: [0.011706799268722534, 0.008697673678398132], all client disparities: [0.057246387004852295, 0.012818962335586548], all client accs: [0.7336562275886536, 0.7743250131607056],  alpha_performance: tensor([0.5646, 0.4354], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:27,661 - utils - INFO - stage1_gradient_single_runtime: 0.00228118896484375
2023-09-28 23:25:27,662 - utils - INFO -  epoch: 135, all client loss: [0.6270488500595093, 0.5644503831863403], all pred client disparities: [0.010231852531433105, 0.0082559734582901], all client disparities: [0.057246387004852295, 0.013539433479309082], all client accs: [0.7336562275886536, 0.7744805216789246],  alpha_performance: tensor([0.5698, 0.4302], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:27,883 - utils - INFO - stage1_gradient_single_runtime: 0.0020678043365478516
2023-09-28 23:25:27,884 - utils - INFO -  epoch: 136, all client loss: [0.6277461647987366, 0.563757598400116], all pred client disparities: [0.008716404438018799, 0.007839828729629517], all client disparities: [0.057246387004852295, 0.013936251401901245], all client accs: [0.7336562275886536, 0.7768756747245789],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:28,107 - utils - INFO - stage1_gradient_single_runtime: 0.0022161006927490234
2023-09-28 23:25:28,108 - utils - INFO -  epoch: 137, all client loss: [0.6257801651954651, 0.5634840726852417], all pred client disparities: [0.011088043451309204, 0.008464142680168152], all client disparities: [0.057246387004852295, 0.013612508773803711], all client accs: [0.7336562275886536, 0.774418294429779],  alpha_performance: tensor([0.5678, 0.4322], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:28,329 - utils - INFO - stage1_gradient_single_runtime: 0.0020787715911865234
2023-09-28 23:25:28,330 - utils - INFO -  epoch: 138, all client loss: [0.6264746189117432, 0.5627942681312561], all pred client disparities: [0.009616881608963013, 0.008039146661758423], all client disparities: [0.057246387004852295, 0.01214030385017395], all client accs: [0.7336562275886536, 0.7769690155982971],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:28,554 - utils - INFO - stage1_gradient_single_runtime: 0.0023217201232910156
2023-09-28 23:25:28,555 - utils - INFO -  epoch: 139, all client loss: [0.624539315700531, 0.5625203847885132], all pred client disparities: [0.01193171739578247, 0.008655428886413574], all client disparities: [0.057246387004852295, 0.013758689165115356], all client accs: [0.7336562275886536, 0.7742627859115601],  alpha_performance: tensor([0.5655, 0.4345], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:28,642 - utils - INFO - valid: True, epoch: 139, loss: [0.6380419731140137, 0.5611371397972107], accuracy: [0.6906077861785889, 0.7809316515922546], mean_accuracy:0.7357697188854218,variance_accuracy:0.045161932706832886, disparity: [0.013636350631713867, 0.009528189897537231], mean_disparity:0.01158227026462555,variance_disparity:0.002054080367088318, pred_disparity: [0.033937305212020874, 0.00027295947074890137]
2023-09-28 23:25:28,768 - utils - INFO - global_valid: True, epoch: 139,  global_loss: 0.5619921088218689, global_accuracy: 0.8031528796334804,  global_disparity:0.011347830295562744, global_pred_disparity: 0.0013092756271362305,
2023-09-28 23:25:28,988 - utils - INFO - stage1_gradient_single_runtime: 0.0022895336151123047
2023-09-28 23:25:28,989 - utils - INFO -  epoch: 140, all client loss: [0.6252298951148987, 0.5618345141410828], all pred client disparities: [0.010505527257919312, 0.008222505450248718], all client disparities: [0.057246387004852295, 0.01141984760761261], all client accs: [0.7336562275886536, 0.7768445611000061],  alpha_performance: tensor([0.5707, 0.4293], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:29,214 - utils - INFO - stage1_gradient_single_runtime: 0.00205230712890625
2023-09-28 23:25:29,214 - utils - INFO -  epoch: 141, all client loss: [0.6259080767631531, 0.561160683631897], all pred client disparities: [0.009039372205734253, 0.007814168930053711], all client disparities: [0.057246387004852295, 0.01408243179321289], all client accs: [0.7336562275886536, 0.7769379019737244],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:29,440 - utils - INFO - stage1_gradient_single_runtime: 0.002299070358276367
2023-09-28 23:25:29,441 - utils - INFO -  epoch: 142, all client loss: [0.623971164226532, 0.5608994960784912], all pred client disparities: [0.011373966932296753, 0.008430227637290955], all client disparities: [0.057246387004852295, 0.011566027998924255], all client accs: [0.7336562275886536, 0.7767512798309326],  alpha_performance: tensor([0.5684, 0.4316], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:29,663 - utils - INFO - stage1_gradient_single_runtime: 0.002092599868774414
2023-09-28 23:25:29,664 - utils - INFO -  epoch: 143, all client loss: [0.6246463656425476, 0.5602287650108337], all pred client disparities: [0.009952515363693237, 0.008013129234313965], all client disparities: [0.057246387004852295, 0.013435065746307373], all client accs: [0.7336562275886536, 0.7768756747245789],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:29,889 - utils - INFO - stage1_gradient_single_runtime: 0.002254962921142578
2023-09-28 23:25:29,889 - utils - INFO -  epoch: 144, all client loss: [0.6227399706840515, 0.5599669218063354], all pred client disparities: [0.012227565050125122, 0.00862133502960205], all client disparities: [0.04818841814994812, 0.011712208390235901], all client accs: [0.7457627654075623, 0.7768445611000061],  alpha_performance: tensor([0.5658, 0.4342], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:30,114 - utils - INFO - stage1_gradient_single_runtime: 0.0022492408752441406
2023-09-28 23:25:30,114 - utils - INFO -  epoch: 145, all client loss: [0.6234111785888672, 0.5593001842498779], all pred client disparities: [0.010851353406906128, 0.00819629430770874], all client disparities: [0.04818841814994812, 0.011492937803268433], all client accs: [0.7457627654075623, 0.7768135070800781],  alpha_performance: tensor([0.5712, 0.4288], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:30,338 - utils - INFO - stage1_gradient_single_runtime: 0.0020666122436523438
2023-09-28 23:25:30,339 - utils - INFO -  epoch: 146, all client loss: [0.6240707635879517, 0.5586448907852173], all pred client disparities: [0.009435713291168213, 0.007794857025146484], all client disparities: [0.057246387004852295, 0.014949068427085876], all client accs: [0.7336562275886536, 0.7769379019737244],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:30,562 - utils - INFO - stage1_gradient_single_runtime: 0.002264261245727539
2023-09-28 23:25:30,563 - utils - INFO -  epoch: 147, all client loss: [0.6221635937690735, 0.5583950281143188], all pred client disparities: [0.011727839708328247, 0.008402571082115173], all client disparities: [0.046376824378967285, 0.011492937803268433], all client accs: [0.7481840252876282, 0.7767823934555054],  alpha_performance: tensor([0.5686, 0.4314], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:30,787 - utils - INFO - stage1_gradient_single_runtime: 0.0022356510162353516
2023-09-28 23:25:30,788 - utils - INFO -  epoch: 148, all client loss: [0.6228199601173401, 0.5577429533004761], all pred client disparities: [0.010357201099395752, 0.00799250602722168], all client disparities: [0.046376824378967285, 0.013508155941963196], all client accs: [0.7481840252876282, 0.7768756747245789],  alpha_performance: tensor([0.5738, 0.4262], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:31,015 - utils - INFO - stage1_gradient_single_runtime: 0.0020596981048583984
2023-09-28 23:25:31,015 - utils - INFO -  epoch: 149, all client loss: [0.623464822769165, 0.557102382183075], all pred client disparities: [0.008948296308517456, 0.007605403661727905], all client disparities: [0.04818841814994812, 0.011138036847114563], all client accs: [0.7457627654075623, 0.7767201662063599],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:31,239 - utils - INFO - stage1_gradient_single_runtime: 0.0022478103637695312
2023-09-28 23:25:31,239 - utils - INFO -  epoch: 150, all client loss: [0.621558427810669, 0.5568634271621704], all pred client disparities: [0.011255532503128052, 0.008212029933929443], all client disparities: [0.046376824378967285, 0.013581231236457825], all client accs: [0.7481840252876282, 0.7767823934555054],  alpha_performance: tensor([0.5712, 0.4288], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:31,463 - utils - INFO - stage1_gradient_single_runtime: 0.002049684524536133
2023-09-28 23:25:31,463 - utils - INFO -  epoch: 151, all client loss: [0.6222005486488342, 0.5562254190444946], all pred client disparities: [0.009891360998153687, 0.007815614342689514], all client disparities: [0.046376824378967285, 0.010709911584854126], all client accs: [0.7481840252876282, 0.7767201662063599],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:31,686 - utils - INFO - stage1_gradient_single_runtime: 0.0022735595703125
2023-09-28 23:25:31,687 - utils - INFO -  epoch: 152, all client loss: [0.6203255653381348, 0.5559852719306946], all pred client disparities: [0.012135565280914307, 0.008414581418037415], all client disparities: [0.046376824378967285, 0.011211007833480835], all client accs: [0.7481840252876282, 0.7770001292228699],  alpha_performance: tensor([0.5684, 0.4316], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:31,912 - utils - INFO - stage1_gradient_single_runtime: 0.0022749900817871094
2023-09-28 23:25:31,913 - utils - INFO -  epoch: 153, all client loss: [0.6209642291069031, 0.5553507208824158], all pred client disparities: [0.010816693305969238, 0.008009687066078186], all client disparities: [0.046376824378967285, 0.01408243179321289], all client accs: [0.7481840252876282, 0.7767201662063599],  alpha_performance: tensor([0.5736, 0.4264], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:32,134 - utils - INFO - stage1_gradient_single_runtime: 0.0021457672119140625
2023-09-28 23:25:32,135 - utils - INFO -  epoch: 154, all client loss: [0.621592104434967, 0.5547269582748413], all pred client disparities: [0.009459972381591797, 0.007626950740814209], all client disparities: [0.046376824378967285, 0.011200681328773499], all client accs: [0.7481840252876282, 0.7765646576881409],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:32,359 - utils - INFO - stage1_gradient_single_runtime: 0.0023152828216552734
2023-09-28 23:25:32,360 - utils - INFO -  epoch: 155, all client loss: [0.6197184920310974, 0.5544973015785217], all pred client disparities: [0.011716783046722412, 0.008224695920944214], all client disparities: [0.046376824378967285, 0.013153120875358582], all client accs: [0.7481840252876282, 0.7768756747245789],  alpha_performance: tensor([0.5708, 0.4292], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:32,591 - utils - INFO - stage1_gradient_single_runtime: 0.0022668838500976562
2023-09-28 23:25:32,592 - utils - INFO -  epoch: 156, all client loss: [0.6203434467315674, 0.5538763403892517], all pred client disparities: [0.010405212640762329, 0.007832854986190796], all client disparities: [0.046376824378967285, 0.01019829511642456], all client accs: [0.7481840252876282, 0.7765335440635681],  alpha_performance: tensor([0.5759, 0.4241], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:32,816 - utils - INFO - stage1_gradient_single_runtime: 0.0020742416381835938
2023-09-28 23:25:32,817 - utils - INFO -  epoch: 157, all client loss: [0.6209577918052673, 0.5532660484313965], all pred client disparities: [0.009056806564331055, 0.007462635636329651], all client disparities: [0.046376824378967285, 0.011346861720085144], all client accs: [0.7481840252876282, 0.7766579389572144],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:33,041 - utils - INFO - stage1_gradient_single_runtime: 0.002748727798461914
2023-09-28 23:25:33,042 - utils - INFO -  epoch: 158, all client loss: [0.619086742401123, 0.5530459880828857], all pred client disparities: [0.01132422685623169, 0.008058682084083557], all client disparities: [0.046376824378967285, 0.010417565703392029], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([0.5731, 0.4269], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:33,278 - utils - INFO - stage1_gradient_single_runtime: 0.002866029739379883
2023-09-28 23:25:33,279 - utils - INFO -  epoch: 159, all client loss: [0.6196986436843872, 0.5524380207061768], all pred client disparities: [0.010020673274993896, 0.007678806781768799], all client disparities: [0.046376824378967285, 0.01077258586883545], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5781, 0.4219], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:33,364 - utils - INFO - valid: True, epoch: 159, loss: [0.6341918110847473, 0.5511001944541931], accuracy: [0.6961326003074646, 0.7752174139022827], mean_accuracy:0.7356750071048737,variance_accuracy:0.03954240679740906, disparity: [0.022727280855178833, 0.017929330468177795], mean_disparity:0.020328305661678314,variance_disparity:0.002398975193500519, pred_disparity: [0.03569298982620239, 0.0006401091814041138]
2023-09-28 23:25:33,506 - utils - INFO - global_valid: True, epoch: 159,  global_loss: 0.5520238876342773, global_accuracy: 0.8038267843540441,  global_disparity:0.019337758421897888, global_pred_disparity: 0.0017362385988235474,
2023-09-28 23:25:33,728 - utils - INFO - stage1_gradient_single_runtime: 0.002084493637084961
2023-09-28 23:25:33,729 - utils - INFO -  epoch: 160, all client loss: [0.6203001737594604, 0.5518403053283691], all pred client disparities: [0.008681237697601318, 0.007319942116737366], all client disparities: [0.046376824378967285, 0.0007022470235824585], all client accs: [0.7481840252876282, 0.7723653316497803],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:33,966 - utils - INFO - stage1_gradient_single_runtime: 0.00229644775390625
2023-09-28 23:25:33,967 - utils - INFO -  epoch: 161, all client loss: [0.6184329390525818, 0.551629364490509], all pred client disparities: [0.010957568883895874, 0.007913962006568909], all client disparities: [0.046376824378967285, 0.010271385312080383], all client accs: [0.7481840252876282, 0.7765957117080688],  alpha_performance: tensor([0.5753, 0.4247], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:34,189 - utils - INFO - stage1_gradient_single_runtime: 0.002104043960571289
2023-09-28 23:25:34,190 - utils - INFO -  epoch: 162, all client loss: [0.6190325021743774, 0.5510337352752686], all pred client disparities: [0.009662866592407227, 0.007544979453086853], all client disparities: [0.046376824378967285, 0.011419951915740967], all client accs: [0.7481840252876282, 0.7766890525817871],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:34,415 - utils - INFO - stage1_gradient_single_runtime: 0.002263307571411133
2023-09-28 23:25:34,416 - utils - INFO -  epoch: 163, all client loss: [0.6171976923942566, 0.5508206486701965], all pred client disparities: [0.011871814727783203, 0.008131757378578186], all client disparities: [0.046376824378967285, 0.01107536256313324], all client accs: [0.7481840252876282, 0.7767823934555054],  alpha_performance: tensor([0.5721, 0.4279], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:34,641 - utils - INFO - stage1_gradient_single_runtime: 0.002287626266479492
2023-09-28 23:25:34,642 - utils - INFO -  epoch: 164, all client loss: [0.6177944540977478, 0.5502278208732605], all pred client disparities: [0.010622173547744751, 0.007753551006317139], all client disparities: [0.046376824378967285, 0.010845676064491272], all client accs: [0.7481840252876282, 0.7766890525817871],  alpha_performance: tensor([0.5772, 0.4228], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:34,869 - utils - INFO - stage1_gradient_single_runtime: 0.0020599365234375
2023-09-28 23:25:34,869 - utils - INFO -  epoch: 165, all client loss: [0.6183813214302063, 0.5496446490287781], all pred client disparities: [0.009337037801742554, 0.007395714521408081], all client disparities: [0.046376824378967285, 0.000556066632270813], all client accs: [0.7481840252876282, 0.7723342180252075],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:35,096 - utils - INFO - stage1_gradient_single_runtime: 0.002248525619506836
2023-09-28 23:25:35,096 - utils - INFO -  epoch: 166, all client loss: [0.6165505647659302, 0.549440324306488], all pred client disparities: [0.011552631855010986, 0.007980391383171082], all client disparities: [0.046376824378967285, 0.011002272367477417], all client accs: [0.7481840252876282, 0.7768756747245789],  alpha_performance: tensor([0.5741, 0.4259], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:35,322 - utils - INFO - stage1_gradient_single_runtime: 0.0022346973419189453
2023-09-28 23:25:35,323 - utils - INFO -  epoch: 167, all client loss: [0.6171351075172424, 0.5488595962524414], all pred client disparities: [0.010312378406524658, 0.007612854242324829], all client disparities: [0.046376824378967285, 0.0012765228748321533], all client accs: [0.7481840252876282, 0.7723031044006348],  alpha_performance: tensor([0.5791, 0.4209], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:35,550 - utils - INFO - stage1_gradient_single_runtime: 0.0020825862884521484
2023-09-28 23:25:35,550 - utils - INFO -  epoch: 168, all client loss: [0.6177099943161011, 0.5482884049415588], all pred client disparities: [0.009037494659423828, 0.007265180349349976], all client disparities: [0.046376824378967285, 0.0008484125137329102], all client accs: [0.7481840252876282, 0.7723342180252075],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:35,776 - utils - INFO - stage1_gradient_single_runtime: 0.0022416114807128906
2023-09-28 23:25:35,777 - utils - INFO -  epoch: 169, all client loss: [0.6158844232559204, 0.5480919480323792], all pred client disparities: [0.01125788688659668, 0.007847458124160767], all client disparities: [0.046376824378967285, 0.011138036847114563], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5759, 0.4241], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:36,005 - utils - INFO - stage1_gradient_single_runtime: 0.002227783203125
2023-09-28 23:25:36,005 - utils - INFO -  epoch: 170, all client loss: [0.6164572834968567, 0.547522783279419], all pred client disparities: [0.010027766227722168, 0.007489696145057678], all client disparities: [0.046376824378967285, 0.001349613070487976], all client accs: [0.7481840252876282, 0.7722408771514893],  alpha_performance: tensor([0.5808, 0.4192], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:36,181 - utils - INFO - stage1_gradient_single_runtime: 0.0020821094512939453
2023-09-28 23:25:36,182 - utils - INFO -  epoch: 171, all client loss: [0.6170207262039185, 0.5469629764556885], all pred client disparities: [0.008763790130615234, 0.0071513354778289795], all client disparities: [0.046376824378967285, 0.001239880919456482], all client accs: [0.7481840252876282, 0.7723031044006348],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:36,408 - utils - INFO - stage1_gradient_single_runtime: 0.002245187759399414
2023-09-28 23:25:36,409 - utils - INFO -  epoch: 172, all client loss: [0.6152012348175049, 0.5467740297317505], all pred client disparities: [0.010987251996994019, 0.007730931043624878], all client disparities: [0.046376824378967285, 0.001057252287864685], all client accs: [0.7481840252876282, 0.7723031044006348],  alpha_performance: tensor([0.5777, 0.4223], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:36,641 - utils - INFO - stage1_gradient_single_runtime: 0.002069711685180664
2023-09-28 23:25:36,642 - utils - INFO -  epoch: 173, all client loss: [0.6157630085945129, 0.546215832233429], all pred client disparities: [0.00976794958114624, 0.00738215446472168], all client disparities: [0.046376824378967285, 0.0014957934617996216], all client accs: [0.7481840252876282, 0.772271990776062],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:36,876 - utils - INFO - stage1_gradient_single_runtime: 0.002290487289428711
2023-09-28 23:25:36,877 - utils - INFO -  epoch: 174, all client loss: [0.6139763593673706, 0.5460243225097656], all pred client disparities: [0.011920064687728882, 0.007954955101013184], all client disparities: [0.046376824378967285, 0.011795833706855774], all client accs: [0.7481840252876282, 0.7768135070800781],  alpha_performance: tensor([0.5742, 0.4258], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:37,111 - utils - INFO - stage1_gradient_single_runtime: 0.0022559165954589844
2023-09-28 23:25:37,112 - utils - INFO -  epoch: 175, all client loss: [0.6145356297492981, 0.5454687476158142], all pred client disparities: [0.01074555516242981, 0.0075965821743011475], all client disparities: [0.046376824378967285, 0.001349613070487976], all client accs: [0.7481840252876282, 0.7722097635269165],  alpha_performance: tensor([0.5792, 0.4208], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:37,340 - utils - INFO - stage1_gradient_single_runtime: 0.0025298595428466797
2023-09-28 23:25:37,341 - utils - INFO -  epoch: 176, all client loss: [0.6150860786437988, 0.5449218153953552], all pred client disparities: [0.009537577629089355, 0.007257059216499329], all client disparities: [0.046376824378967285, 0.0005925148725509644], all client accs: [0.7481840252876282, 0.7723342180252075],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:37,572 - utils - INFO - stage1_gradient_single_runtime: 0.002823352813720703
2023-09-28 23:25:37,574 - utils - INFO -  epoch: 177, all client loss: [0.6133055090904236, 0.5447375178337097], all pred client disparities: [0.011690706014633179, 0.007827222347259521], all client disparities: [0.046376824378967285, 0.0001070946455001831], all client accs: [0.7481840252876282, 0.772551953792572],  alpha_performance: tensor([0.5757, 0.4243], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:37,812 - utils - INFO - stage1_gradient_single_runtime: 0.0022840499877929688
2023-09-28 23:25:37,813 - utils - INFO -  epoch: 178, all client loss: [0.6138538122177124, 0.5441928505897522], all pred client disparities: [0.010527431964874268, 0.007477790117263794], all client disparities: [0.046376824378967285, 0.0006291568279266357], all client accs: [0.7481840252876282, 0.7723653316497803],  alpha_performance: tensor([0.5807, 0.4193], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:38,038 - utils - INFO - stage1_gradient_single_runtime: 0.0021021366119384766
2023-09-28 23:25:38,039 - utils - INFO -  epoch: 179, all client loss: [0.6143932938575745, 0.5436567068099976], all pred client disparities: [0.009331345558166504, 0.007146731019020081], all client disparities: [0.046376824378967285, 0.002033427357673645], all client accs: [0.7481840252876282, 0.7723653316497803],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:38,123 - utils - INFO - valid: True, epoch: 179, loss: [0.6282451152801514, 0.5427062511444092], accuracy: [0.6961326003074646, 0.7752174139022827], mean_accuracy:0.7356750071048737,variance_accuracy:0.03954240679740906, disparity: [0.022727280855178833, 0.017189696431159973], mean_disparity:0.019958488643169403,variance_disparity:0.00276879221200943, pred_disparity: [0.032933205366134644, 6.0111284255981445e-05]
2023-09-28 23:25:38,248 - utils - INFO - global_valid: True, epoch: 179,  global_loss: 0.5436572432518005, global_accuracy: 0.8051690213969636,  global_disparity:0.018621429800987244, global_pred_disparity: 0.0012692511081695557,
2023-09-28 23:25:38,470 - utils - INFO - stage1_gradient_single_runtime: 0.0023355484008789062
2023-09-28 23:25:38,470 - utils - INFO -  epoch: 180, all client loss: [0.6126196980476379, 0.5434791445732117], all pred client disparities: [0.011483907699584961, 0.007714062929153442], all client disparities: [0.046376824378967285, 0.00024791061878204346], all client accs: [0.7481840252876282, 0.772551953792572],  alpha_performance: tensor([0.5771, 0.4229], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:38,695 - utils - INFO - stage1_gradient_single_runtime: 0.00223541259765625
2023-09-28 23:25:38,696 - utils - INFO -  epoch: 181, all client loss: [0.6131573915481567, 0.5429449081420898], all pred client disparities: [0.010332435369491577, 0.007372811436653137], all client disparities: [0.046376824378967285, 1.8209218978881836e-05], all client accs: [0.7481840252876282, 0.7723963856697083],  alpha_performance: tensor([0.5820, 0.4180], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:38,919 - utils - INFO - stage1_gradient_single_runtime: 0.0020813941955566406
2023-09-28 23:25:38,920 - utils - INFO -  epoch: 182, all client loss: [0.6136865615844727, 0.5424191355705261], all pred client disparities: [0.009148865938186646, 0.0070495158433914185], all client disparities: [0.046376824378967285, 0.00048813968896865845], all client accs: [0.7481840252876282, 0.772147536277771],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:39,143 - utils - INFO - stage1_gradient_single_runtime: 0.0022482872009277344
2023-09-28 23:25:39,143 - utils - INFO -  epoch: 183, all client loss: [0.6119207143783569, 0.5422477722167969], all pred client disparities: [0.011299163103103638, 0.007613837718963623], all client disparities: [0.046376824378967285, 0.000895276665687561], all client accs: [0.7481840252876282, 0.772551953792572],  alpha_performance: tensor([0.5785, 0.4215], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:39,364 - utils - INFO - stage1_gradient_single_runtime: 0.002273082733154297
2023-09-28 23:25:39,365 - utils - INFO -  epoch: 184, all client loss: [0.6124482750892639, 0.5417236685752869], all pred client disparities: [0.010160088539123535, 0.007280081510543823], all client disparities: [0.046376824378967285, 0.0013860464096069336], all client accs: [0.7481840252876282, 0.7724586129188538],  alpha_performance: tensor([0.5832, 0.4168], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:39,587 - utils - INFO - stage1_gradient_single_runtime: 0.002079486846923828
2023-09-28 23:25:39,588 - utils - INFO -  epoch: 185, all client loss: [0.6129675507545471, 0.5412076115608215], all pred client disparities: [0.008989423513412476, 0.006963908672332764], all client disparities: [0.046376824378967285, 0.00012268871068954468], all client accs: [0.7481840252876282, 0.7720853686332703],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:39,807 - utils - INFO - stage1_gradient_single_runtime: 0.0022325515747070312
2023-09-28 23:25:39,808 - utils - INFO -  epoch: 186, all client loss: [0.6112101674079895, 0.5410420894622803], all pred client disparities: [0.011135995388031006, 0.007525041699409485], all client disparities: [0.04456523060798645, 0.0007490962743759155], all client accs: [0.7506053447723389, 0.7725830674171448],  alpha_performance: tensor([0.5796, 0.4204], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:40,023 - utils - INFO - stage1_gradient_single_runtime: 0.0022346973419189453
2023-09-28 23:25:40,024 - utils - INFO -  epoch: 187, all client loss: [0.6117280125617981, 0.5405275821685791], all pred client disparities: [0.01000964641571045, 0.007198214530944824], all client disparities: [0.04456523060798645, 0.002033427357673645], all client accs: [0.7506053447723389, 0.7725208401679993],  alpha_performance: tensor([0.5844, 0.4156], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:40,241 - utils - INFO - stage1_gradient_single_runtime: 0.0020771026611328125
2023-09-28 23:25:40,241 - utils - INFO -  epoch: 188, all client loss: [0.6122378706932068, 0.5400210618972778], all pred client disparities: [0.008852481842041016, 0.00688856840133667], all client disparities: [0.04456523060798645, 0.00012268871068954468], all client accs: [0.7506053447723389, 0.772147536277771],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:40,463 - utils - INFO - stage1_gradient_single_runtime: 0.002256155014038086
2023-09-28 23:25:40,464 - utils - INFO -  epoch: 189, all client loss: [0.6104894876480103, 0.5398609638214111], all pred client disparities: [0.010993778705596924, 0.007446393370628357], all client disparities: [0.04456523060798645, 0.0006760060787200928], all client accs: [0.7457627654075623, 0.7726763486862183],  alpha_performance: tensor([0.5807, 0.4193], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:40,686 - utils - INFO - stage1_gradient_single_runtime: 0.0020990371704101562
2023-09-28 23:25:40,687 - utils - INFO -  epoch: 190, all client loss: [0.6109980940818787, 0.5393556952476501], all pred client disparities: [0.00988072156906128, 0.007125958800315857], all client disparities: [0.04456523060798645, 0.0016679763793945312], all client accs: [0.7506053447723389, 0.7724586129188538],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:40,908 - utils - INFO - stage1_gradient_single_runtime: 0.0022237300872802734
2023-09-28 23:25:40,908 - utils - INFO -  epoch: 191, all client loss: [0.6092830896377563, 0.5391924977302551], all pred client disparities: [0.011945575475692749, 0.007677778601646423], all client disparities: [0.04456523060798645, 0.00010173022747039795], all client accs: [0.7457627654075623, 0.7724897265434265],  alpha_performance: tensor([0.5768, 0.4232], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:41,132 - utils - INFO - stage1_gradient_single_runtime: 0.0022346973419189453
2023-09-28 23:25:41,133 - utils - INFO -  epoch: 192, all client loss: [0.6097896695137024, 0.5386891961097717], all pred client disparities: [0.010876566171646118, 0.00734730064868927], all client disparities: [0.04456523060798645, 0.0021169334650039673], all client accs: [0.7457627654075623, 0.772831916809082],  alpha_performance: tensor([0.5817, 0.4183], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:41,355 - utils - INFO - stage1_gradient_single_runtime: 0.0020918846130371094
2023-09-28 23:25:41,355 - utils - INFO -  epoch: 193, all client loss: [0.6102885603904724, 0.5381934642791748], all pred client disparities: [0.009777247905731201, 0.007033616304397583], all client disparities: [0.04456523060798645, 0.0005246847867965698], all client accs: [0.7506053447723389, 0.772116482257843],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:41,579 - utils - INFO - stage1_gradient_single_runtime: 0.002225637435913086
2023-09-28 23:25:41,580 - utils - INFO -  epoch: 194, all client loss: [0.6085824370384216, 0.53803551197052], all pred client disparities: [0.01183512806892395, 0.00758226215839386], all client disparities: [0.04456523060798645, 2.8640031814575195e-05], all client accs: [0.7457627654075623, 0.772551953792572],  alpha_performance: tensor([0.5777, 0.4223], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:41,804 - utils - INFO - stage1_gradient_single_runtime: 0.002284526824951172
2023-09-28 23:25:41,804 - utils - INFO -  epoch: 195, all client loss: [0.6090794801712036, 0.5375416874885559], all pred client disparities: [0.010779738426208496, 0.007258310914039612], all client disparities: [0.04456523060798645, 0.0021169334650039673], all client accs: [0.7457627654075623, 0.7729251980781555],  alpha_performance: tensor([0.5825, 0.4175], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:42,035 - utils - INFO - stage1_gradient_single_runtime: 0.002027273178100586
2023-09-28 23:25:42,036 - utils - INFO -  epoch: 196, all client loss: [0.6095691323280334, 0.5370551943778992], all pred client disparities: [0.009694606065750122, 0.006950840353965759], all client disparities: [0.04456523060798645, 0.0001331120729446411], all client accs: [0.7506053447723389, 0.7723342180252075],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:42,258 - utils - INFO - stage1_gradient_single_runtime: 0.0022802352905273438
2023-09-28 23:25:42,259 - utils - INFO -  epoch: 197, all client loss: [0.6078723669052124, 0.5369021892547607], all pred client disparities: [0.011743992567062378, 0.007496178150177002], all client disparities: [0.04456523060798645, 0.0007490962743759155], all client accs: [0.7457627654075623, 0.772707462310791],  alpha_performance: tensor([0.5785, 0.4215], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:42,479 - utils - INFO - stage1_gradient_single_runtime: 0.0022072792053222656
2023-09-28 23:25:42,480 - utils - INFO -  epoch: 198, all client loss: [0.6083604097366333, 0.5364173650741577], all pred client disparities: [0.01070261001586914, 0.00717829167842865], all client disparities: [0.04456523060798645, 0.0018245875835418701], all client accs: [0.7457627654075623, 0.7728630304336548],  alpha_performance: tensor([0.5832, 0.4168], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:42,704 - utils - INFO - stage1_gradient_single_runtime: 0.002036571502685547
2023-09-28 23:25:42,704 - utils - INFO -  epoch: 199, all client loss: [0.608841061592102, 0.5359397530555725], all pred client disparities: [0.00963205099105835, 0.006876498460769653], all client disparities: [0.04456523060798645, 0.00020620226860046387], all client accs: [0.7506053447723389, 0.7723963856697083],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:42,786 - utils - INFO - valid: True, epoch: 199, loss: [0.6240784525871277, 0.5349889397621155], accuracy: [0.6906077861785889, 0.7752174139022827], mean_accuracy:0.7329126000404358,variance_accuracy:0.042304813861846924, disparity: [0.03181818127632141, 0.014552995562553406], mean_disparity:0.02318558841943741,variance_disparity:0.008632592856884003, pred_disparity: [0.03309682011604309, 2.530217170715332e-05]
2023-09-28 23:25:42,911 - utils - INFO - global_valid: True, epoch: 199,  global_loss: 0.5359793305397034, global_accuracy: 0.8056186710677111,  global_disparity:0.015812426805496216, global_pred_disparity: 0.0013832002878189087,
2023-09-28 23:25:43,130 - utils - INFO - stage1_gradient_single_runtime: 0.002238035202026367
2023-09-28 23:25:43,131 - utils - INFO -  epoch: 200, all client loss: [0.6071543097496033, 0.5357913970947266], all pred client disparities: [0.011671513319015503, 0.007418453693389893], all client disparities: [0.04456523060798645, 0.00219002366065979], all client accs: [0.7457627654075623, 0.7727385759353638],  alpha_performance: tensor([0.5791, 0.4209], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:43,351 - utils - INFO - stage1_gradient_single_runtime: 0.002242565155029297
2023-09-28 23:25:43,352 - utils - INFO -  epoch: 201, all client loss: [0.6076335906982422, 0.535315215587616], all pred client disparities: [0.010644525289535522, 0.007106170058250427], all client disparities: [0.04456523060798645, 0.0004411637783050537], all client accs: [0.7457627654075623, 0.772427499294281],  alpha_performance: tensor([0.5839, 0.4161], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:43,579 - utils - INFO - stage1_gradient_single_runtime: 0.002058744430541992
2023-09-28 23:25:43,580 - utils - INFO -  epoch: 202, all client loss: [0.6081056594848633, 0.5348461866378784], all pred client disparities: [0.00958898663520813, 0.006809622049331665], all client disparities: [0.04456523060798645, 0.0001331120729446411], all client accs: [0.7506053447723389, 0.7723653316497803],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:43,804 - utils - INFO - stage1_gradient_single_runtime: 0.002275228500366211
2023-09-28 23:25:43,805 - utils - INFO -  epoch: 203, all client loss: [0.606429398059845, 0.5347020626068115], all pred client disparities: [0.011617153882980347, 0.007348105311393738], all client disparities: [0.04456523060798645, 0.002263113856315613], all client accs: [0.7457627654075623, 0.772831916809082],  alpha_performance: tensor([0.5797, 0.4203], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:44,028 - utils - INFO - stage1_gradient_single_runtime: 0.002261638641357422
2023-09-28 23:25:44,029 - utils - INFO -  epoch: 204, all client loss: [0.6069002151489258, 0.5342342853546143], all pred client disparities: [0.010604947805404663, 0.007040947675704956], all client disparities: [0.04456523060798645, 0.0003523826599121094], all client accs: [0.7457627654075623, 0.772427499294281],  alpha_performance: tensor([0.5844, 0.4156], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:44,262 - utils - INFO - stage1_gradient_single_runtime: 0.0020797252655029297
2023-09-28 23:25:44,263 - utils - INFO -  epoch: 205, all client loss: [0.6073640584945679, 0.5337734222412109], all pred client disparities: [0.009564757347106934, 0.0067492276430130005], all client disparities: [0.04456523060798645, 0.0001331120729446411], all client accs: [0.7506053447723389, 0.7723963856697083],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:44,496 - utils - INFO - stage1_gradient_single_runtime: 0.0022614002227783203
2023-09-28 23:25:44,497 - utils - INFO -  epoch: 206, all client loss: [0.6056987643241882, 0.5336334109306335], all pred client disparities: [0.01158025860786438, 0.007284209132194519], all client disparities: [0.04094204306602478, 0.0011772215366363525], all client accs: [0.7409201264381409, 0.772707462310791],  alpha_performance: tensor([0.5802, 0.4198], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:44,725 - utils - INFO - stage1_gradient_single_runtime: 0.002411365509033203
2023-09-28 23:25:44,725 - utils - INFO -  epoch: 207, all client loss: [0.60616135597229, 0.5331737399101257], all pred client disparities: [0.010583192110061646, 0.006981790065765381], all client disparities: [0.04094204306602478, 0.0002792924642562866], all client accs: [0.7409201264381409, 0.7724586129188538],  alpha_performance: tensor([0.5848, 0.4152], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:44,949 - utils - INFO - stage1_gradient_single_runtime: 0.002234220504760742
2023-09-28 23:25:44,950 - utils - INFO -  epoch: 208, all client loss: [0.6066171526908875, 0.5327208638191223], all pred client disparities: [0.009558558464050293, 0.0066945552825927734], all client disparities: [0.04094204306602478, 0.00020620226860046387], all client accs: [0.7409201264381409, 0.7724586129188538],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:45,174 - utils - INFO - stage1_gradient_single_runtime: 0.0022263526916503906
2023-09-28 23:25:45,175 - utils - INFO -  epoch: 209, all client loss: [0.6049633026123047, 0.5325846076011658], all pred client disparities: [0.011560230515897274, 0.007225930690765381], all client disparities: [0.04094204306602478, 0.0018976777791976929], all client accs: [0.7409201264381409, 0.7726763486862183],  alpha_performance: tensor([0.5805, 0.4195], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:45,403 - utils - INFO - stage1_gradient_single_runtime: 0.0022563934326171875
2023-09-28 23:25:45,404 - utils - INFO -  epoch: 210, all client loss: [0.6054179668426514, 0.5321328043937683], all pred client disparities: [0.010578572750091553, 0.006927892565727234], all client disparities: [0.04094204306602478, 0.0004411637783050537], all client accs: [0.7409201264381409, 0.772427499294281],  alpha_performance: tensor([0.5852, 0.4148], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:45,633 - utils - INFO - stage1_gradient_single_runtime: 0.0024871826171875
2023-09-28 23:25:45,635 - utils - INFO -  epoch: 211, all client loss: [0.605866014957428, 0.5316875576972961], all pred client disparities: [0.009569913148880005, 0.006644755601882935], all client disparities: [0.04094204306602478, 0.0008535757660865784], all client accs: [0.7409201264381409, 0.7724586129188538],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:45,862 - utils - INFO - stage1_gradient_single_runtime: 0.002276182174682617
2023-09-28 23:25:45,863 - utils - INFO -  epoch: 212, all client loss: [0.604223906993866, 0.5315548777580261], all pred client disparities: [0.011556446552276611, 0.0071725547313690186], all client disparities: [0.04094204306602478, 0.001323401927947998], all client accs: [0.7409201264381409, 0.7728008031845093],  alpha_performance: tensor([0.5808, 0.4192], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:46,095 - utils - INFO - stage1_gradient_single_runtime: 0.0022199153900146484
2023-09-28 23:25:46,096 - utils - INFO -  epoch: 213, all client loss: [0.6046709418296814, 0.5311106443405151], all pred client disparities: [0.01059049367904663, 0.006878584623336792], all client disparities: [0.04094204306602478, 0.0002949833869934082], all client accs: [0.7409201264381409, 0.7724897265434265],  alpha_performance: tensor([0.5854, 0.4146], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:46,322 - utils - INFO - stage1_gradient_single_runtime: 0.002093791961669922
2023-09-28 23:25:46,323 - utils - INFO -  epoch: 214, all client loss: [0.6051114797592163, 0.5306729674339294], all pred client disparities: [0.00959804654121399, 0.0065991878509521484], all client disparities: [0.04094204306602478, 0.0008535757660865784], all client accs: [0.7409201264381409, 0.7724897265434265],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:46,548 - utils - INFO - stage1_gradient_single_runtime: 0.0022673606872558594
2023-09-28 23:25:46,549 - utils - INFO -  epoch: 215, all client loss: [0.6034815311431885, 0.5305435061454773], all pred client disparities: [0.011568278074264526, 0.007123380899429321], all client disparities: [0.04094204306602478, 0.001323401927947998], all client accs: [0.7409201264381409, 0.7728008031845093],  alpha_performance: tensor([0.5810, 0.4190], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:46,776 - utils - INFO - stage1_gradient_single_runtime: 0.0022389888763427734
2023-09-28 23:25:46,777 - utils - INFO -  epoch: 216, all client loss: [0.6039210557937622, 0.5301067233085632], all pred client disparities: [0.010618269443511963, 0.006833165884017944], all client disparities: [0.04094204306602478, 0.00042547285556793213], all client accs: [0.7409201264381409, 0.7724586129188538],  alpha_performance: tensor([0.5855, 0.4145], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:47,006 - utils - INFO - stage1_gradient_single_runtime: 0.0020682811737060547
2023-09-28 23:25:47,007 - utils - INFO -  epoch: 217, all client loss: [0.6043543219566345, 0.5296761989593506], all pred client disparities: [0.009642302989959717, 0.006557241082191467], all client disparities: [0.04094204306602478, 0.00020620226860046387], all client accs: [0.7409201264381409, 0.772427499294281],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:47,238 - utils - INFO - stage1_gradient_single_runtime: 0.0022735595703125
2023-09-28 23:25:47,239 - utils - INFO -  epoch: 218, all client loss: [0.60273677110672, 0.5295498371124268], all pred client disparities: [0.011595070362091064, 0.007077842950820923], all client disparities: [0.04094204306602478, 0.0009423494338989258], all client accs: [0.7409201264381409, 0.7723653316497803],  alpha_performance: tensor([0.5810, 0.4190], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:47,466 - utils - INFO - stage1_gradient_single_runtime: 0.002350330352783203
2023-09-28 23:25:47,467 - utils - INFO -  epoch: 219, all client loss: [0.6031690835952759, 0.5291202664375305], all pred client disparities: [0.010661303997039795, 0.006791055202484131], all client disparities: [0.04094204306602478, 0.0003523826599121094], all client accs: [0.7409201264381409, 0.772427499294281],  alpha_performance: tensor([0.5856, 0.4144], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:47,555 - utils - INFO - valid: True, epoch: 219, loss: [0.6214173436164856, 0.527865469455719], accuracy: [0.6850829124450684, 0.7762733101844788], mean_accuracy:0.7306781113147736,variance_accuracy:0.0455951988697052, disparity: [0.03181818127632141, 0.013826370239257812], mean_disparity:0.022822275757789612,variance_disparity:0.0089959055185318, pred_disparity: [0.035544008016586304, 0.0006818026304244995]
2023-09-28 23:25:47,685 - utils - INFO - global_valid: True, epoch: 219,  global_loss: 0.52890545129776, global_accuracy: 0.8055961619244826,  global_disparity:0.015124261379241943, global_pred_disparity: 0.002048805356025696,
2023-09-28 23:25:47,911 - utils - INFO - stage1_gradient_single_runtime: 0.0020809173583984375
2023-09-28 23:25:47,912 - utils - INFO -  epoch: 220, all client loss: [0.6035952568054199, 0.5286967754364014], all pred client disparities: [0.009702086448669434, 0.006518348585814238], all client disparities: [0.04094204306602478, 0.00020620226860046387], all client accs: [0.7409201264381409, 0.7723963856697083],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:48,140 - utils - INFO - stage1_gradient_single_runtime: 0.0026111602783203125
2023-09-28 23:25:48,141 - utils - INFO -  epoch: 221, all client loss: [0.6019904017448425, 0.5285732746124268], all pred client disparities: [0.011636227369308472, 0.007035315036773682], all client disparities: [0.039130449295043945, 0.0009423494338989258], all client accs: [0.7409201264381409, 0.772427499294281],  alpha_performance: tensor([0.5810, 0.4190], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:48,378 - utils - INFO - stage1_gradient_single_runtime: 0.002488374710083008
2023-09-28 23:25:48,379 - utils - INFO -  epoch: 222, all client loss: [0.6024157404899597, 0.5281506776809692], all pred client disparities: [0.01071891188621521, 0.006751731038093567], all client disparities: [0.04094204306602478, 0.0005142539739608765], all client accs: [0.7409201264381409, 0.772427499294281],  alpha_performance: tensor([0.5855, 0.4145], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:48,628 - utils - INFO - stage1_gradient_single_runtime: 0.0028045177459716797
2023-09-28 23:25:48,630 - utils - INFO -  epoch: 223, all client loss: [0.602834939956665, 0.527734100818634], all pred client disparities: [0.009776651859283447, 0.006481975317001343], all client disparities: [0.04094204306602478, 0.00020620226860046387], all client accs: [0.7409201264381409, 0.772427499294281],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:48,880 - utils - INFO - stage1_gradient_single_runtime: 0.0022568702697753906
2023-09-28 23:25:48,882 - utils - INFO -  epoch: 224, all client loss: [0.6012430787086487, 0.5276132822036743], all pred client disparities: [0.011691153049468994, 0.006995350122451782], all client disparities: [0.039130449295043945, 0.0010154396295547485], all client accs: [0.7409201264381409, 0.7723963856697083],  alpha_performance: tensor([0.5809, 0.4191], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:49,111 - utils - INFO - stage1_gradient_single_runtime: 0.0025887489318847656
2023-09-28 23:25:49,112 - utils - INFO -  epoch: 225, all client loss: [0.6016615033149719, 0.527197539806366], all pred client disparities: [0.010790526866912842, 0.0067147016525268555], all client disparities: [0.039130449295043945, 0.0004411637783050537], all client accs: [0.7409201264381409, 0.7723653316497803],  alpha_performance: tensor([0.5854, 0.4146], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:49,344 - utils - INFO - stage1_gradient_single_runtime: 0.0020542144775390625
2023-09-28 23:25:49,344 - utils - INFO -  epoch: 226, all client loss: [0.6020740270614624, 0.5267875790596008], all pred client disparities: [0.009865343570709229, 0.006447702646255493], all client disparities: [0.04094204306602478, 0.0001069977879524231], all client accs: [0.7409201264381409, 0.7723653316497803],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:49,569 - utils - INFO - stage1_gradient_single_runtime: 0.0022652149200439453
2023-09-28 23:25:49,570 - utils - INFO -  epoch: 227, all client loss: [0.6004953980445862, 0.5266692638397217], all pred client disparities: [0.011759161949157715, 0.006957441568374634], all client disparities: [0.039130449295043945, 0.0010154396295547485], all client accs: [0.7409201264381409, 0.7723963856697083],  alpha_performance: tensor([0.5806, 0.4194], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:49,798 - utils - INFO - stage1_gradient_single_runtime: 0.002268075942993164
2023-09-28 23:25:49,799 - utils - INFO -  epoch: 228, all client loss: [0.6009070873260498, 0.5262601375579834], all pred client disparities: [0.010875433683395386, 0.006679549813270569], all client disparities: [0.039130449295043945, 0.0004411637783050537], all client accs: [0.7409201264381409, 0.7723963856697083],  alpha_performance: tensor([0.5851, 0.4149], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:50,027 - utils - INFO - stage1_gradient_single_runtime: 0.0020432472229003906
2023-09-28 23:25:50,028 - utils - INFO -  epoch: 229, all client loss: [0.601313054561615, 0.5258567929267883], all pred client disparities: [0.009967595338821411, 0.006415039300918579], all client disparities: [0.039130449295043945, 3.390759229660034e-05], all client accs: [0.7409201264381409, 0.772427499294281],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:50,306 - utils - INFO - stage1_gradient_single_runtime: 0.00376129150390625
2023-09-28 23:25:50,307 - utils - INFO -  epoch: 230, all client loss: [0.5997477173805237, 0.5257408022880554], all pred client disparities: [0.01183977723121643, 0.00692121684551239], all client disparities: [0.039130449295043945, 0.001161620020866394], all client accs: [0.7384988069534302, 0.7723342180252075],  alpha_performance: tensor([0.5803, 0.4197], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:50,513 - utils - INFO - stage1_gradient_single_runtime: 0.0022699832916259766
2023-09-28 23:25:50,514 - utils - INFO -  epoch: 231, all client loss: [0.6001529097557068, 0.5253381729125977], all pred client disparities: [0.010972976684570312, 0.006645858287811279], all client disparities: [0.039130449295043945, 0.0004411637783050537], all client accs: [0.7409201264381409, 0.7723653316497803],  alpha_performance: tensor([0.5848, 0.4152], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:50,788 - utils - INFO - stage1_gradient_single_runtime: 0.0026428699493408203
2023-09-28 23:25:50,788 - utils - INFO -  epoch: 232, all client loss: [0.600552499294281, 0.5249411463737488], all pred client disparities: [0.01008269190788269, 0.006383642554283142], all client disparities: [0.039130449295043945, 0.0007543712854385376], all client accs: [0.7409201264381409, 0.7724586129188538],  alpha_performance: tensor([0.5890, 0.4110], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:51,019 - utils - INFO - stage1_gradient_single_runtime: 0.0022695064544677734
2023-09-28 23:25:51,020 - utils - INFO -  epoch: 233, all client loss: [0.6009464859962463, 0.524549663066864], all pred client disparities: [0.009168952703475952, 0.006134286522865295], all client disparities: [0.039130449295043945, 0.0010415613651275635], all client accs: [0.7409201264381409, 0.772707462310791],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:51,262 - utils - INFO - stage1_gradient_single_runtime: 0.0025403499603271484
2023-09-28 23:25:51,263 - utils - INFO -  epoch: 234, all client loss: [0.5993761420249939, 0.5244417786598206], all pred client disparities: [0.011078715324401855, 0.006637081503868103], all client disparities: [0.035507261753082275, 0.0004411637783050537], all client accs: [0.7433414459228516, 0.7723653316497803],  alpha_performance: tensor([0.5844, 0.4156], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:51,498 - utils - INFO - stage1_gradient_single_runtime: 0.0022699832916259766
2023-09-28 23:25:51,499 - utils - INFO -  epoch: 235, all client loss: [0.5997698903083801, 0.5240504145622253], all pred client disparities: [0.010205954313278198, 0.006376326084136963], all client disparities: [0.035507261753082275, 0.0007543712854385376], all client accs: [0.7457627654075623, 0.7725208401679993],  alpha_performance: tensor([0.5886, 0.4114], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:51,731 - utils - INFO - stage1_gradient_single_runtime: 0.0020935535430908203
2023-09-28 23:25:51,732 - utils - INFO -  epoch: 236, all client loss: [0.6001582741737366, 0.5236644744873047], all pred client disparities: [0.009310245513916016, 0.006128281354904175], all client disparities: [0.039130449295043945, 0.0013965815305709839], all client accs: [0.7409201264381409, 0.7726452350616455],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:51,967 - utils - INFO - stage1_gradient_single_runtime: 0.002767324447631836
2023-09-28 23:25:51,968 - utils - INFO -  epoch: 237, all client loss: [0.5986027121543884, 0.5235582590103149], all pred client disparities: [0.011195361614227295, 0.006627500057220459], all client disparities: [0.035507261753082275, 0.0004411637783050537], all client accs: [0.7433414459228516, 0.7723653316497803],  alpha_performance: tensor([0.5839, 0.4161], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:52,201 - utils - INFO - stage1_gradient_single_runtime: 0.0022656917572021484
2023-09-28 23:25:52,202 - utils - INFO -  epoch: 238, all client loss: [0.5989908576011658, 0.5231725573539734], all pred client disparities: [0.010340392589569092, 0.006368115544319153], all client disparities: [0.035507261753082275, 0.0003993585705757141], all client accs: [0.7433414459228516, 0.7725208401679993],  alpha_performance: tensor([0.5881, 0.4119], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:52,427 - utils - INFO - stage1_gradient_single_runtime: 0.0020570755004882812
2023-09-28 23:25:52,427 - utils - INFO -  epoch: 239, all client loss: [0.5993736982345581, 0.5227921605110168], all pred client disparities: [0.009462833404541016, 0.0061212629079818726], all client disparities: [0.035507261753082275, 0.0006030350923538208], all client accs: [0.7457627654075623, 0.7727385759353638],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:52,512 - utils - INFO - valid: True, epoch: 239, loss: [0.6171139478683472, 0.5218371748924255], accuracy: [0.6906077861785889, 0.7759627103805542], mean_accuracy:0.7332852482795715,variance_accuracy:0.042677462100982666, disparity: [0.0363636314868927, 0.010610982775688171], mean_disparity:0.023487307131290436,variance_disparity:0.012876324355602264, pred_disparity: [0.03427305817604065, 0.00042037665843963623]
2023-09-28 23:25:52,636 - utils - INFO - global_valid: True, epoch: 239,  global_loss: 0.5228963494300842, global_accuracy: 0.8063445987779198,  global_disparity:0.011913642287254333, global_pred_disparity: 0.0018995553255081177,
2023-09-28 23:25:52,857 - utils - INFO - stage1_gradient_single_runtime: 0.0022542476654052734
2023-09-28 23:25:52,858 - utils - INFO -  epoch: 240, all client loss: [0.5978330373764038, 0.5226874947547913], all pred client disparities: [0.011322498321533203, 0.006616950035095215], all client disparities: [0.035507261753082275, 0.0006812885403633118], all client accs: [0.7433414459228516, 0.7723031044006348],  alpha_performance: tensor([0.5833, 0.4167], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:53,080 - utils - INFO - stage1_gradient_single_runtime: 0.002261638641357422
2023-09-28 23:25:53,081 - utils - INFO -  epoch: 241, all client loss: [0.598215639591217, 0.5223073363304138], all pred client disparities: [0.010485291481018066, 0.00635884702205658], all client disparities: [0.035507261753082275, 0.00032626837491989136], all client accs: [0.7433414459228516, 0.772551953792572],  alpha_performance: tensor([0.5875, 0.4125], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:53,305 - utils - INFO - stage1_gradient_single_runtime: 0.0020749568939208984
2023-09-28 23:25:53,306 - utils - INFO -  epoch: 242, all client loss: [0.5985929369926453, 0.5219323635101318], all pred client disparities: [0.009626030921936035, 0.006113067269325256], all client disparities: [0.035507261753082275, 0.0006761178374290466], all client accs: [0.7457627654075623, 0.7727385759353638],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:53,532 - utils - INFO - stage1_gradient_single_runtime: 0.0023047924041748047
2023-09-28 23:25:53,533 - utils - INFO -  epoch: 243, all client loss: [0.5970672965049744, 0.5218291878700256], all pred client disparities: [0.01145946979522705, 0.006605252623558044], all client disparities: [0.035507261753082275, 0.001401752233505249], all client accs: [0.7433414459228516, 0.7723031044006348],  alpha_performance: tensor([0.5826, 0.4174], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:53,758 - utils - INFO - stage1_gradient_single_runtime: 0.002288818359375
2023-09-28 23:25:53,759 - utils - INFO -  epoch: 244, all client loss: [0.5974443554878235, 0.5214544534683228], all pred client disparities: [0.010640114545822144, 0.006348341703414917], all client disparities: [0.035507261753082275, 0.00032626837491989136], all client accs: [0.7433414459228516, 0.772551953792572],  alpha_performance: tensor([0.5869, 0.4131], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:53,983 - utils - INFO - stage1_gradient_single_runtime: 0.0020570755004882812
2023-09-28 23:25:53,983 - utils - INFO -  epoch: 245, all client loss: [0.5978163480758667, 0.5210849046707153], all pred client disparities: [0.009799212217330933, 0.006103545427322388], all client disparities: [0.035507261753082275, 0.0006761178374290466], all client accs: [0.7433414459228516, 0.7727696895599365],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:54,208 - utils - INFO - stage1_gradient_single_runtime: 0.0022630691528320312
2023-09-28 23:25:54,209 - utils - INFO -  epoch: 246, all client loss: [0.5963056683540344, 0.5209831595420837], all pred client disparities: [0.011605709791183472, 0.006592303514480591], all client disparities: [0.035507261753082275, 0.001401752233505249], all client accs: [0.7433414459228516, 0.7723031044006348],  alpha_performance: tensor([0.5818, 0.4182], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:54,434 - utils - INFO - stage1_gradient_single_runtime: 0.0022745132446289062
2023-09-28 23:25:54,435 - utils - INFO -  epoch: 247, all client loss: [0.596677303314209, 0.5206138491630554], all pred client disparities: [0.01080438494682312, 0.00633646547794342], all client disparities: [0.035507261753082275, 0.00032626837491989136], all client accs: [0.7433414459228516, 0.7724897265434265],  alpha_performance: tensor([0.5861, 0.4139], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:54,658 - utils - INFO - stage1_gradient_single_runtime: 0.002094745635986328
2023-09-28 23:25:54,659 - utils - INFO -  epoch: 248, all client loss: [0.5970439314842224, 0.5202494859695435], all pred client disparities: [0.009981811046600342, 0.00609259307384491], all client disparities: [0.035507261753082275, 4.43384051322937e-05], all client accs: [0.7433414459228516, 0.7726452350616455],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:54,886 - utils - INFO - stage1_gradient_single_runtime: 0.0022423267364501953
2023-09-28 23:25:54,887 - utils - INFO -  epoch: 249, all client loss: [0.5955482125282288, 0.5201489925384521], all pred client disparities: [0.011760801076889038, 0.0065779536962509155], all client disparities: [0.035507261753082275, 0.0013286620378494263], all client accs: [0.7433414459228516, 0.772116482257843],  alpha_performance: tensor([0.5809, 0.4191], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:55,113 - utils - INFO - stage1_gradient_single_runtime: 0.0022611618041992188
2023-09-28 23:25:55,114 - utils - INFO -  epoch: 250, all client loss: [0.5959144830703735, 0.5197851061820984], all pred client disparities: [0.010977417230606079, 0.006323069334030151], all client disparities: [0.035507261753082275, 0.0003993585705757141], all client accs: [0.7433414459228516, 0.7725830674171448],  alpha_performance: tensor([0.5852, 0.4148], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:55,341 - utils - INFO - stage1_gradient_single_runtime: 0.0022699832916259766
2023-09-28 23:25:55,342 - utils - INFO -  epoch: 251, all client loss: [0.5962759852409363, 0.5194259881973267], all pred client disparities: [0.010173320770263672, 0.00608004629611969], all client disparities: [0.035507261753082275, 0.00026360154151916504], all client accs: [0.7433414459228516, 0.7727385759353638],  alpha_performance: tensor([0.5893, 0.4107], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:55,568 - utils - INFO - stage1_gradient_single_runtime: 0.0020880699157714844
2023-09-28 23:25:55,569 - utils - INFO -  epoch: 252, all client loss: [0.5966324210166931, 0.519071638584137], all pred client disparities: [0.00934860110282898, 0.0058485716581344604], all client disparities: [0.035507261753082275, 0.0022527948021888733], all client accs: [0.7433414459228516, 0.773827314376831],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:55,794 - utils - INFO - stage1_gradient_single_runtime: 0.0022368431091308594
2023-09-28 23:25:55,795 - utils - INFO -  epoch: 253, all client loss: [0.5951347947120667, 0.5189774036407471], all pred client disparities: [0.011154979467391968, 0.006330057978630066], all client disparities: [0.035507261753082275, 0.00032626837491989136], all client accs: [0.7409201264381409, 0.7723342180252075],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:56,018 - utils - INFO - stage1_gradient_single_runtime: 0.0022127628326416016
2023-09-28 23:25:56,019 - utils - INFO -  epoch: 254, all client loss: [0.5954915285110474, 0.5186229944229126], all pred client disparities: [0.010369420051574707, 0.006087183952331543], all client disparities: [0.035507261753082275, 0.0004568547010421753], all client accs: [0.7409201264381409, 0.772707462310791],  alpha_performance: tensor([0.5883, 0.4117], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:56,244 - utils - INFO - stage1_gradient_single_runtime: 0.0020873546600341797
2023-09-28 23:25:56,244 - utils - INFO -  epoch: 255, all client loss: [0.5958434343338013, 0.5182732939720154], all pred client disparities: [0.009563535451889038, 0.005855754017829895], all client disparities: [0.035507261753082275, 0.0022527948021888733], all client accs: [0.7409201264381409, 0.773827314376831],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:56,469 - utils - INFO - stage1_gradient_single_runtime: 0.00223541259765625
2023-09-28 23:25:56,469 - utils - INFO -  epoch: 256, all client loss: [0.5943617224693298, 0.5181798934936523], all pred client disparities: [0.011340022087097168, 0.00633394718170166], all client disparities: [0.035507261753082275, 0.0010467320680618286], all client accs: [0.7409201264381409, 0.7723031044006348],  alpha_performance: tensor([0.5832, 0.4168], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:56,692 - utils - INFO - stage1_gradient_single_runtime: 0.002225160598754883
2023-09-28 23:25:56,693 - utils - INFO -  epoch: 257, all client loss: [0.5947137475013733, 0.5178301334381104], all pred client disparities: [0.01057279109954834, 0.0060912370681762695], all client disparities: [0.035507261753082275, 0.000529944896697998], all client accs: [0.7409201264381409, 0.7726763486862183],  alpha_performance: tensor([0.5873, 0.4127], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:56,918 - utils - INFO - stage1_gradient_single_runtime: 0.002052783966064453
2023-09-28 23:25:56,919 - utils - INFO -  epoch: 258, all client loss: [0.5950611233711243, 0.5174849629402161], all pred client disparities: [0.00978580117225647, 0.005859851837158203], all client disparities: [0.035507261753082275, 0.0022527948021888733], all client accs: [0.7409201264381409, 0.7736406326293945],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:57,146 - utils - INFO - stage1_gradient_single_runtime: 0.0022857189178466797
2023-09-28 23:25:57,147 - utils - INFO -  epoch: 259, all client loss: [0.5935953855514526, 0.5173924565315247], all pred client disparities: [0.011531710624694824, 0.0063347965478897095], all client disparities: [0.035507261753082275, 2.8751792342518456e-05], all client accs: [0.7409201264381409, 0.7722097635269165],  alpha_performance: tensor([0.5821, 0.4179], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:57,233 - utils - INFO - valid: True, epoch: 259, loss: [0.6142717599868774, 0.5161764621734619], accuracy: [0.6906077861785889, 0.7762733101844788], mean_accuracy:0.7334405481815338,variance_accuracy:0.042832762002944946, disparity: [0.0363636314868927, 0.00886186957359314], mean_disparity:0.02261275053024292,variance_disparity:0.01375088095664978, pred_disparity: [0.035167843103408813, 0.0008148550987243652]
2023-09-28 23:25:57,359 - utils - INFO - global_valid: True, epoch: 259,  global_loss: 0.517267107963562, global_accuracy: 0.8065432709789292,  global_disparity:0.010250791907310486, global_pred_disparity: 0.0022973716259002686,
2023-09-28 23:25:57,581 - utils - INFO - stage1_gradient_single_runtime: 0.0022575855255126953
2023-09-28 23:25:57,582 - utils - INFO -  epoch: 260, all client loss: [0.5939427018165588, 0.5170472264289856], all pred client disparities: [0.010782986879348755, 0.006092235445976257], all client disparities: [0.035507261753082275, 0.000529944896697998], all client accs: [0.7409201264381409, 0.7725208401679993],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:57,806 - utils - INFO - stage1_gradient_single_runtime: 0.002223968505859375
2023-09-28 23:25:57,807 - utils - INFO -  epoch: 261, all client loss: [0.5942854285240173, 0.5167067050933838], all pred client disparities: [0.010014861822128296, 0.005860835313796997], all client disparities: [0.035507261753082275, 0.0016054287552833557], all client accs: [0.7409201264381409, 0.7734851241111755],  alpha_performance: tensor([0.5902, 0.4098], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:58,031 - utils - INFO - stage1_gradient_single_runtime: 0.002138376235961914
2023-09-28 23:25:58,031 - utils - INFO -  epoch: 262, all client loss: [0.5946236848831177, 0.5163706541061401], all pred client disparities: [0.009227365255355835, 0.005640342831611633], all client disparities: [0.035507261753082275, 0.002106614410877228], all client accs: [0.7409201264381409, 0.7743250131607056],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:58,263 - utils - INFO - stage1_gradient_single_runtime: 0.002208232879638672
2023-09-28 23:25:58,264 - utils - INFO -  epoch: 263, all client loss: [0.5931577086448669, 0.5162834525108337], all pred client disparities: [0.01099589467048645, 0.006111219525337219], all client disparities: [0.035507261753082275, 0.000529944896697998], all client accs: [0.7409201264381409, 0.7725208401679993],  alpha_performance: tensor([0.5850, 0.4150], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:58,493 - utils - INFO - stage1_gradient_single_runtime: 0.0022802352905273438
2023-09-28 23:25:58,494 - utils - INFO -  epoch: 264, all client loss: [0.5934963226318359, 0.5159468650817871], all pred client disparities: [0.010246574878692627, 0.005879268050193787], all client disparities: [0.035507261753082275, 0.0016054287552833557], all client accs: [0.7409201264381409, 0.7733917832374573],  alpha_performance: tensor([0.5890, 0.4110], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:58,725 - utils - INFO - stage1_gradient_single_runtime: 0.002057790756225586
2023-09-28 23:25:58,726 - utils - INFO -  epoch: 265, all client loss: [0.5938304662704468, 0.5156148076057434], all pred client disparities: [0.009478360414505005, 0.005658090114593506], all client disparities: [0.035507261753082275, 0.0021797046065330505], all client accs: [0.7409201264381409, 0.7740139365196228],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:58,955 - utils - INFO - stage1_gradient_single_runtime: 0.0022406578063964844
2023-09-28 23:25:58,956 - utils - INFO -  epoch: 266, all client loss: [0.5923812389373779, 0.5155280232429504], all pred client disparities: [0.011214345693588257, 0.006125897169113159], all client disparities: [0.035507261753082275, 0.00011742115020751953], all client accs: [0.7409201264381409, 0.7725208401679993],  alpha_performance: tensor([0.5838, 0.4162], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:59,183 - utils - INFO - stage1_gradient_single_runtime: 0.0022673606872558594
2023-09-28 23:25:59,184 - utils - INFO -  epoch: 267, all client loss: [0.5927157402038574, 0.5151956677436829], all pred client disparities: [0.010483741760253906, 0.005893424153327942], all client disparities: [0.035507261753082275, 0.0009580552577972412], all client accs: [0.7409201264381409, 0.7733917832374573],  alpha_performance: tensor([0.5877, 0.4123], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:59,417 - utils - INFO - stage1_gradient_single_runtime: 0.002065896987915039
2023-09-28 23:25:59,418 - utils - INFO -  epoch: 268, all client loss: [0.5930458903312683, 0.5148676633834839], all pred client disparities: [0.009734749794006348, 0.0056716203689575195], all client disparities: [0.035507261753082275, 0.0006656944751739502], all client accs: [0.7409201264381409, 0.7737339735031128],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:59,644 - utils - INFO - stage1_gradient_single_runtime: 0.002288341522216797
2023-09-28 23:25:59,645 - utils - INFO -  epoch: 269, all client loss: [0.5916131138801575, 0.5147812366485596], all pred client disparities: [0.011437773704528809, 0.006136417388916016], all client disparities: [0.03369566798210144, 0.00038376450538635254], all client accs: [0.7409201264381409, 0.7733607292175293],  alpha_performance: tensor([0.5824, 0.4176], device='cuda:0', dtype=torch.float64)
2023-09-28 23:25:59,870 - utils - INFO - stage1_gradient_single_runtime: 0.002253293991088867
2023-09-28 23:25:59,871 - utils - INFO -  epoch: 270, all client loss: [0.5919435620307922, 0.5144529342651367], all pred client disparities: [0.010725945234298706, 0.005903422832489014], all client disparities: [0.03369566798210144, 0.0011042281985282898], all client accs: [0.7409201264381409, 0.7734540104866028],  alpha_performance: tensor([0.5864, 0.4136], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:00,095 - utils - INFO - stage1_gradient_single_runtime: 0.0020737648010253906
2023-09-28 23:26:00,095 - utils - INFO -  epoch: 271, all client loss: [0.5922696590423584, 0.5141288638114929], all pred client disparities: [0.009995996952056885, 0.0056810081005096436], all client disparities: [0.035507261753082275, 0.0006656944751739502], all client accs: [0.7409201264381409, 0.7736717462539673],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:00,321 - utils - INFO - stage1_gradient_single_runtime: 0.002746105194091797
2023-09-28 23:26:00,321 - utils - INFO -  epoch: 272, all client loss: [0.590853214263916, 0.514042854309082], all pred client disparities: [0.011665821075439453, 0.00614282488822937], all client disparities: [0.031884074211120605, 0.00019051134586334229], all client accs: [0.7433414459228516, 0.7733917832374573],  alpha_performance: tensor([0.5810, 0.4190], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:00,551 - utils - INFO - stage1_gradient_single_runtime: 0.002627134323120117
2023-09-28 23:26:00,552 - utils - INFO -  epoch: 273, all client loss: [0.5911795496940613, 0.5137186646461487], all pred client disparities: [0.010972648859024048, 0.005909323692321777], all client disparities: [0.031884074211120605, 0.0004568547010421753], all client accs: [0.7433414459228516, 0.77342289686203],  alpha_performance: tensor([0.5849, 0.4151], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:00,792 - utils - INFO - stage1_gradient_single_runtime: 0.002231121063232422
2023-09-28 23:26:00,792 - utils - INFO -  epoch: 274, all client loss: [0.5915016531944275, 0.5133985877037048], all pred client disparities: [0.010261684656143188, 0.005686327815055847], all client disparities: [0.031884074211120605, 0.0001645013689994812], all client accs: [0.7433414459228516, 0.77370285987854],  alpha_performance: tensor([0.5888, 0.4112], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:01,021 - utils - INFO - stage1_gradient_single_runtime: 0.0022134780883789062
2023-09-28 23:26:01,022 - utils - INFO -  epoch: 275, all client loss: [0.5918195247650146, 0.5130826830863953], all pred client disparities: [0.009533166885375977, 0.00547356903553009], all client disparities: [0.031884074211120605, 0.0029732584953308105], all client accs: [0.7433414459228516, 0.773827314376831],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:01,254 - utils - INFO - stage1_gradient_single_runtime: 0.0022737979888916016
2023-09-28 23:26:01,255 - utils - INFO -  epoch: 276, all client loss: [0.5904041528701782, 0.5130013823509216], all pred client disparities: [0.011219918727874756, 0.005931288003921509], all client disparities: [0.031884074211120605, 0.0004568547010421753], all client accs: [0.7433414459228516, 0.7733607292175293],  alpha_performance: tensor([0.5835, 0.4165], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:01,479 - utils - INFO - stage1_gradient_single_runtime: 0.0022466182708740234
2023-09-28 23:26:01,480 - utils - INFO -  epoch: 277, all client loss: [0.5907226800918579, 0.5126849412918091], all pred client disparities: [0.010527998208999634, 0.005707204341888428], all client disparities: [0.031884074211120605, 9.141117334365845e-05], all client accs: [0.7433414459228516, 0.7736095786094666],  alpha_performance: tensor([0.5873, 0.4127], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:01,710 - utils - INFO - stage1_gradient_single_runtime: 0.0020928382873535156
2023-09-28 23:26:01,711 - utils - INFO -  epoch: 278, all client loss: [0.5910369753837585, 0.512372612953186], all pred client disparities: [0.009818851947784424, 0.005493283271789551], all client disparities: [0.031884074211120605, 0.0022527948021888733], all client accs: [0.7433414459228516, 0.7738584280014038],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:01,937 - utils - INFO - stage1_gradient_single_runtime: 0.00228118896484375
2023-09-28 23:26:01,938 - utils - INFO -  epoch: 279, all client loss: [0.5896384119987488, 0.5122913122177124], all pred client disparities: [0.011470615863800049, 0.00594821572303772], all client disparities: [0.031884074211120605, 0.00026360154151916504], all client accs: [0.7433414459228516, 0.7733296155929565],  alpha_performance: tensor([0.5819, 0.4181], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:02,027 - utils - INFO - valid: True, epoch: 279, loss: [0.6113866567611694, 0.5110908150672913], accuracy: [0.6961326003074646, 0.7771428823471069], mean_accuracy:0.7366377413272858,variance_accuracy:0.04050514101982117, disparity: [0.04090908169746399, 0.013408616185188293], mean_disparity:0.02715884894132614,variance_disparity:0.013750232756137848, pred_disparity: [0.03541263937950134, 0.0010585486888885498]
2023-09-28 23:26:02,155 - utils - INFO - global_valid: True, epoch: 279,  global_loss: 0.512205958366394, global_accuracy: 0.8068882369287534,  global_disparity:0.014464274048805237, global_pred_disparity: 0.0025648772716522217,
2023-09-28 23:26:02,384 - utils - INFO - stage1_gradient_single_runtime: 0.0022847652435302734
2023-09-28 23:26:02,385 - utils - INFO -  epoch: 280, all client loss: [0.5899533033370972, 0.5119785070419312], all pred client disparities: [0.010797470808029175, 0.005723103880882263], all client disparities: [0.031884074211120605, 9.141117334365845e-05], all client accs: [0.7433414459228516, 0.7736717462539673],  alpha_performance: tensor([0.5857, 0.4143], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:02,618 - utils - INFO - stage1_gradient_single_runtime: 0.002285003662109375
2023-09-28 23:26:02,618 - utils - INFO -  epoch: 281, all client loss: [0.5902640223503113, 0.5116697549819946], all pred client disparities: [0.010107547044754028, 0.005508050322532654], all client disparities: [0.031884074211120605, 0.0022527948021888733], all client accs: [0.7433414459228516, 0.77370285987854],  alpha_performance: tensor([0.5895, 0.4105], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:02,852 - utils - INFO - stage1_gradient_single_runtime: 0.0020666122436523438
2023-09-28 23:26:02,853 - utils - INFO -  epoch: 282, all client loss: [0.5905707478523254, 0.5113650560379028], all pred client disparities: [0.009400784969329834, 0.005302831530570984], all client disparities: [0.031884074211120605, 0.002106614410877228], all client accs: [0.7433414459228516, 0.7737962007522583],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:03,090 - utils - INFO - stage1_gradient_single_runtime: 0.0022950172424316406
2023-09-28 23:26:03,091 - utils - INFO -  epoch: 283, all client loss: [0.5891744494438171, 0.5112878680229187], all pred client disparities: [0.011066436767578125, 0.005753651261329651], all client disparities: [0.031884074211120605, 0.00023759156465530396], all client accs: [0.7433414459228516, 0.77370285987854],  alpha_performance: tensor([0.5841, 0.4159], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:03,321 - utils - INFO - stage1_gradient_single_runtime: 0.00225067138671875
2023-09-28 23:26:03,322 - utils - INFO -  epoch: 284, all client loss: [0.5894819498062134, 0.5109822750091553], all pred client disparities: [0.010395586490631104, 0.005537047982215881], all client disparities: [0.031884074211120605, 0.0022527948021888733], all client accs: [0.7433414459228516, 0.7736095786094666],  alpha_performance: tensor([0.5879, 0.4121], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:03,569 - utils - INFO - stage1_gradient_single_runtime: 0.0024983882904052734
2023-09-28 23:26:03,570 - utils - INFO -  epoch: 285, all client loss: [0.5897855758666992, 0.510680615901947], all pred client disparities: [0.00970834493637085, 0.005330204963684082], all client disparities: [0.031884074211120605, 0.002106614410877228], all client accs: [0.7433414459228516, 0.7738584280014038],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:03,819 - utils - INFO - stage1_gradient_single_runtime: 0.002424478530883789
2023-09-28 23:26:03,820 - utils - INFO -  epoch: 286, all client loss: [0.588406503200531, 0.5106034278869629], all pred client disparities: [0.011337578296661377, 0.0057784318923950195], all client disparities: [0.031884074211120605, 0.0017776191234588623], all client accs: [0.7433414459228516, 0.7737962007522583],  alpha_performance: tensor([0.5824, 0.4176], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:04,061 - utils - INFO - stage1_gradient_single_runtime: 0.0022814273834228516
2023-09-28 23:26:04,062 - utils - INFO -  epoch: 287, all client loss: [0.5887107849121094, 0.5103011131286621], all pred client disparities: [0.010685652494430542, 0.005560353398323059], all client disparities: [0.031884074211120605, 0.0016785189509391785], all client accs: [0.7433414459228516, 0.7736406326293945],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:04,300 - utils - INFO - stage1_gradient_single_runtime: 0.002252817153930664
2023-09-28 23:26:04,300 - utils - INFO -  epoch: 288, all client loss: [0.5890111327171326, 0.5100026726722717], all pred client disparities: [0.010017752647399902, 0.005351990461349487], all client disparities: [0.031884074211120605, 0.0014592483639717102], all client accs: [0.7433414459228516, 0.7738894820213318],  alpha_performance: tensor([0.5898, 0.4102], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:04,531 - utils - INFO - stage1_gradient_single_runtime: 0.0020928382873535156
2023-09-28 23:26:04,532 - utils - INFO -  epoch: 289, all client loss: [0.5893076062202454, 0.509708046913147], all pred client disparities: [0.00933372974395752, 0.005153104662895203], all client disparities: [0.031884074211120605, 0.002106614410877228], all client accs: [0.7433414459228516, 0.773827314376831],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:04,761 - utils - INFO - stage1_gradient_single_runtime: 0.0023005008697509766
2023-09-28 23:26:04,762 - utils - INFO -  epoch: 290, all client loss: [0.5879315733909607, 0.5096344947814941], all pred client disparities: [0.010974079370498657, 0.005597203969955444], all client disparities: [0.031884074211120605, 0.00038376450538635254], all client accs: [0.7433414459228516, 0.7736406326293945],  alpha_performance: tensor([0.5844, 0.4156], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:04,935 - utils - INFO - stage1_gradient_single_runtime: 0.0022411346435546875
2023-09-28 23:26:04,936 - utils - INFO -  epoch: 291, all client loss: [0.5882290601730347, 0.5093388557434082], all pred client disparities: [0.010325312614440918, 0.005386918783187866], all client disparities: [0.031884074211120605, 0.001532338559627533], all client accs: [0.7433414459228516, 0.77370285987854],  alpha_performance: tensor([0.5881, 0.4119], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:05,164 - utils - INFO - stage1_gradient_single_runtime: 0.002153158187866211
2023-09-28 23:26:05,165 - utils - INFO -  epoch: 292, all client loss: [0.588522732257843, 0.5090471506118774], all pred client disparities: [0.009660869836807251, 0.005186036229133606], all client disparities: [0.031884074211120605, 0.0021797046065330505], all client accs: [0.7433414459228516, 0.7737650871276855],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:05,391 - utils - INFO - stage1_gradient_single_runtime: 0.002245664596557617
2023-09-28 23:26:05,392 - utils - INFO -  epoch: 293, all client loss: [0.5871642827987671, 0.508973240852356], all pred client disparities: [0.011263549327850342, 0.005627751350402832], all client disparities: [0.031884074211120605, 0.00038376450538635254], all client accs: [0.7433414459228516, 0.7736406326293945],  alpha_performance: tensor([0.5826, 0.4174], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:05,619 - utils - INFO - stage1_gradient_single_runtime: 0.0022351741790771484
2023-09-28 23:26:05,620 - utils - INFO -  epoch: 294, all client loss: [0.5874588489532471, 0.5086806416511536], all pred client disparities: [0.010633796453475952, 0.005415633320808411], all client disparities: [0.031884074211120605, 9.141117334365845e-05], all client accs: [0.7433414459228516, 0.77370285987854],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:05,847 - utils - INFO - stage1_gradient_single_runtime: 0.002094745635986328
2023-09-28 23:26:05,848 - utils - INFO -  epoch: 295, all client loss: [0.5877496004104614, 0.5083917379379272], all pred client disparities: [0.0099886953830719, 0.005212873220443726], all client disparities: [0.031884074211120605, 0.0014592483639717102], all client accs: [0.7409201264381409, 0.7737962007522583],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:06,082 - utils - INFO - stage1_gradient_single_runtime: 0.002251148223876953
2023-09-28 23:26:06,082 - utils - INFO -  epoch: 296, all client loss: [0.5864084362983704, 0.5083176493644714], all pred client disparities: [0.011553735472261906, 0.005652248859405518], all client disparities: [0.031884074211120605, 0.00026360154151916504], all client accs: [0.7409201264381409, 0.7735162377357483],  alpha_performance: tensor([0.5806, 0.4194], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:06,309 - utils - INFO - stage1_gradient_single_runtime: 0.0022735595703125
2023-09-28 23:26:06,310 - utils - INFO -  epoch: 297, all client loss: [0.5866999626159668, 0.508027970790863], all pred client disparities: [0.010942816734313965, 0.005438372492790222], all client disparities: [0.031884074211120605, 0.0003106743097305298], all client accs: [0.7409201264381409, 0.77370285987854],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:06,537 - utils - INFO - stage1_gradient_single_runtime: 0.002277374267578125
2023-09-28 23:26:06,538 - utils - INFO -  epoch: 298, all client loss: [0.5869877338409424, 0.5077419877052307], all pred client disparities: [0.0103168785572052, 0.005233824253082275], all client disparities: [0.031884074211120605, 9.141117334365845e-05], all client accs: [0.7409201264381409, 0.7737962007522583],  alpha_performance: tensor([0.5879, 0.4121], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:06,765 - utils - INFO - stage1_gradient_single_runtime: 0.002105712890625
2023-09-28 23:26:06,766 - utils - INFO -  epoch: 299, all client loss: [0.5872718691825867, 0.5074597001075745], all pred client disparities: [0.009676128625869751, 0.005038395524024963], all client disparities: [0.031884074211120605, 0.0022527948021888733], all client accs: [0.7409201264381409, 0.7737650871276855],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:06,851 - utils - INFO - valid: True, epoch: 299, loss: [0.608510434627533, 0.5064870715141296], accuracy: [0.7016574740409851, 0.7772049903869629], mean_accuracy:0.739431232213974,variance_accuracy:0.03777375817298889, disparity: [0.04545453190803528, 0.016045302152633667], mean_disparity:0.030749917030334473,variance_disparity:0.014704614877700806, pred_disparity: [0.03511509299278259, 0.0011735409498214722]
2023-09-28 23:26:06,991 - utils - INFO - global_valid: True, epoch: 299,  global_loss: 0.5076212882995605, global_accuracy: 0.8073328473951182,  global_disparity:0.016843467950820923, global_pred_disparity: 0.002720966935157776,
2023-09-28 23:26:07,212 - utils - INFO - stage1_gradient_single_runtime: 0.0022568702697753906
2023-09-28 23:26:07,212 - utils - INFO -  epoch: 300, all client loss: [0.5859343409538269, 0.5073888301849365], all pred client disparities: [0.011248856782913208, 0.0054737478494644165], all client disparities: [0.031884074211120605, 0.0003106743097305298], all client accs: [0.7409201264381409, 0.773547351360321],  alpha_performance: tensor([0.5823, 0.4177], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:07,439 - utils - INFO - stage1_gradient_single_runtime: 0.002257823944091797
2023-09-28 23:26:07,440 - utils - INFO -  epoch: 301, all client loss: [0.5862194895744324, 0.5071055889129639], all pred client disparities: [0.01064196228981018, 0.005267038941383362], all client disparities: [0.031884074211120605, 0.00023759156465530396], all client accs: [0.7409201264381409, 0.7737339735031128],  alpha_performance: tensor([0.5859, 0.4141], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:07,671 - utils - INFO - stage1_gradient_single_runtime: 0.0028204917907714844
2023-09-28 23:26:07,672 - utils - INFO -  epoch: 302, all client loss: [0.5865009427070618, 0.506825864315033], all pred client disparities: [0.010020524263381958, 0.005069434642791748], all client disparities: [0.031884074211120605, 0.0008118748664855957], all client accs: [0.7409201264381409, 0.7737962007522583],  alpha_performance: tensor([0.5894, 0.4106], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:07,905 - utils - INFO - stage1_gradient_single_runtime: 0.0027039051055908203
2023-09-28 23:26:07,906 - utils - INFO -  epoch: 303, all client loss: [0.5867789387702942, 0.5065496563911438], all pred client disparities: [0.00938454270362854, 0.0048806071281433105], all client disparities: [0.031884074211120605, 0.002033531665802002], all client accs: [0.7409201264381409, 0.77398282289505],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:08,149 - utils - INFO - stage1_gradient_single_runtime: 0.0024178028106689453
2023-09-28 23:26:08,150 - utils - INFO -  epoch: 304, all client loss: [0.585445761680603, 0.5064818263053894], all pred client disparities: [0.010963261127471924, 0.0053119659423828125], all client disparities: [0.031884074211120605, 0.0003106743097305298], all client accs: [0.7409201264381409, 0.773547351360321],  alpha_performance: tensor([0.5839, 0.4161], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:08,384 - utils - INFO - stage1_gradient_single_runtime: 0.0022735595703125
2023-09-28 23:26:08,385 - utils - INFO -  epoch: 305, all client loss: [0.585724949836731, 0.5062044858932495], all pred client disparities: [0.010361045598983765, 0.005111813545227051], all client disparities: [0.031884074211120605, 0.0009580552577972412], all client accs: [0.7409201264381409, 0.773827314376831],  alpha_performance: tensor([0.5874, 0.4126], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:08,611 - utils - INFO - stage1_gradient_single_runtime: 0.0020885467529296875
2023-09-28 23:26:08,611 - utils - INFO -  epoch: 306, all client loss: [0.5860005617141724, 0.5059305429458618], all pred client disparities: [0.009744524955749512, 0.004920467734336853], all client disparities: [0.031884074211120605, 0.0005926117300987244], all client accs: [0.7409201264381409, 0.7740450501441956],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:08,838 - utils - INFO - stage1_gradient_single_runtime: 0.0023605823516845703
2023-09-28 23:26:08,838 - utils - INFO -  epoch: 307, all client loss: [0.584685206413269, 0.5058621764183044], all pred client disparities: [0.011283516883850098, 0.005349844694137573], all client disparities: [0.031884074211120605, 0.0006030350923538208], all client accs: [0.7409201264381409, 0.7736095786094666],  alpha_performance: tensor([0.5818, 0.4182], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:09,064 - utils - INFO - stage1_gradient_single_runtime: 0.0022678375244140625
2023-09-28 23:26:09,065 - utils - INFO -  epoch: 308, all client loss: [0.584962010383606, 0.5055872201919556], all pred client disparities: [0.010700196027755737, 0.005147308111190796], all client disparities: [0.031884074211120605, 0.0012504085898399353], all client accs: [0.7409201264381409, 0.7736717462539673],  alpha_performance: tensor([0.5853, 0.4147], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:09,291 - utils - INFO - stage1_gradient_single_runtime: 0.002260446548461914
2023-09-28 23:26:09,292 - utils - INFO -  epoch: 309, all client loss: [0.5852352380752563, 0.5053156614303589], all pred client disparities: [0.010103046894073486, 0.004953548312187195], all client disparities: [0.031884074211120605, 0.0008849650621414185], all client accs: [0.7409201264381409, 0.7741072773933411],  alpha_performance: tensor([0.5887, 0.4113], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:09,522 - utils - INFO - stage1_gradient_single_runtime: 0.002079010009765625
2023-09-28 23:26:09,523 - utils - INFO -  epoch: 310, all client loss: [0.5855050683021545, 0.5050475597381592], all pred client disparities: [0.009491950273513794, 0.004768341779708862], all client disparities: [0.031884074211120605, 0.00030025094747543335], all client accs: [0.7409201264381409, 0.7739517092704773],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:09,747 - utils - INFO - stage1_gradient_single_runtime: 0.0023429393768310547
2023-09-28 23:26:09,747 - utils - INFO -  epoch: 311, all client loss: [0.584194540977478, 0.5049819350242615], all pred client disparities: [0.01103469729423523, 0.005193829536437988], all client disparities: [0.031884074211120605, 0.000529944896697998], all client accs: [0.7409201264381409, 0.7736717462539673],  alpha_performance: tensor([0.5831, 0.4169], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:09,971 - utils - INFO - stage1_gradient_single_runtime: 0.0022873878479003906
2023-09-28 23:26:09,972 - utils - INFO -  epoch: 312, all client loss: [0.5844656825065613, 0.5047125816345215], all pred client disparities: [0.010456562042236328, 0.004997342824935913], all client disparities: [0.031884074211120605, 0.0009580552577972412], all client accs: [0.7409201264381409, 0.77398282289505],  alpha_performance: tensor([0.5866, 0.4134], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:10,197 - utils - INFO - stage1_gradient_single_runtime: 0.002059459686279297
2023-09-28 23:26:10,198 - utils - INFO -  epoch: 313, all client loss: [0.5847333669662476, 0.5044465661048889], all pred client disparities: [0.009864985942840576, 0.004809394478797913], all client disparities: [0.024637669324874878, 0.0008118748664855957], all client accs: [0.7506053447723389, 0.7740761637687683],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:10,424 - utils - INFO - stage1_gradient_single_runtime: 0.00223541259765625
2023-09-28 23:26:10,425 - utils - INFO -  epoch: 314, all client loss: [0.5834407806396484, 0.5043802857398987], all pred client disparities: [0.01136729121208191, 0.005233079195022583], all client disparities: [0.024637669324874878, 0.000529944896697998], all client accs: [0.7506053447723389, 0.7736095786094666],  alpha_performance: tensor([0.5809, 0.4191], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:10,653 - utils - INFO - stage1_gradient_single_runtime: 0.002288341522216797
2023-09-28 23:26:10,654 - utils - INFO -  epoch: 315, all client loss: [0.5837096571922302, 0.5041131377220154], all pred client disparities: [0.010807931423187256, 0.005033999681472778], all client disparities: [0.024637669324874878, 0.0009580552577972412], all client accs: [0.7506053447723389, 0.7738584280014038],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:10,877 - utils - INFO - stage1_gradient_single_runtime: 0.002260446548461914
2023-09-28 23:26:10,878 - utils - INFO -  epoch: 316, all client loss: [0.5839751362800598, 0.5038493871688843], all pred client disparities: [0.010235399007797241, 0.004843473434448242], all client disparities: [0.024637669324874878, 0.0008849650621414185], all client accs: [0.7506053447723389, 0.7739517092704773],  alpha_performance: tensor([0.5877, 0.4123], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:11,109 - utils - INFO - stage1_gradient_single_runtime: 0.0020542144775390625
2023-09-28 23:26:11,110 - utils - INFO -  epoch: 317, all client loss: [0.584237277507782, 0.5035889148712158], all pred client disparities: [0.009649723768234253, 0.004661247134208679], all client disparities: [0.024637669324874878, 0.0003733411431312561], all client accs: [0.7506053447723389, 0.77398282289505],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:11,342 - utils - INFO - stage1_gradient_single_runtime: 0.0022656917572021484
2023-09-28 23:26:11,343 - utils - INFO -  epoch: 318, all client loss: [0.5829498171806335, 0.5035251379013062], all pred client disparities: [0.011153757572174072, 0.005081146955490112], all client disparities: [0.024637669324874878, 0.0011042281985282898], all client accs: [0.7481840252876282, 0.7735784649848938],  alpha_performance: tensor([0.5821, 0.4179], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:11,577 - utils - INFO - stage1_gradient_single_runtime: 0.002272367477416992
2023-09-28 23:26:11,578 - utils - INFO -  epoch: 319, all client loss: [0.5832133293151855, 0.5032633543014526], all pred client disparities: [0.010600149631500244, 0.00488772988319397], all client disparities: [0.024637669324874878, 0.0008849650621414185], all client accs: [0.7481840252876282, 0.7738894820213318],  alpha_performance: tensor([0.5854, 0.4146], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:11,664 - utils - INFO - valid: True, epoch: 319, loss: [0.6068047881126404, 0.5020843744277954], accuracy: [0.7016574740409851, 0.7773913145065308], mean_accuracy:0.7395243942737579,variance_accuracy:0.03786692023277283, disparity: [0.04545453190803528, 0.014726966619491577], mean_disparity:0.030090749263763428,variance_disparity:0.01536378264427185, pred_disparity: [0.03649872541427612, 0.0018457919359207153]
2023-09-28 23:26:11,792 - utils - INFO - global_valid: True, epoch: 319,  global_loss: 0.5032486319541931, global_accuracy: 0.8073905577855603,  global_disparity:0.015582233667373657, global_pred_disparity: 0.0033528953790664673,
2023-09-28 23:26:12,015 - utils - INFO - stage1_gradient_single_runtime: 0.002243518829345703
2023-09-28 23:26:12,016 - utils - INFO -  epoch: 320, all client loss: [0.5834735035896301, 0.5030048489570618], all pred client disparities: [0.010033667087554932, 0.004702627658843994], all client disparities: [0.024637669324874878, 0.00044643133878707886], all client accs: [0.7481840252876282, 0.7738584280014038],  alpha_performance: tensor([0.5888, 0.4112], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:12,250 - utils - INFO - stage1_gradient_single_runtime: 0.002118825912475586
2023-09-28 23:26:12,250 - utils - INFO -  epoch: 321, all client loss: [0.5837304592132568, 0.5027495622634888], all pred client disparities: [0.00945425033569336, 0.004525601863861084], all client disparities: [0.024637669324874878, 0.0003733411431312561], all client accs: [0.7481840252876282, 0.77398282289505],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:12,483 - utils - INFO - stage1_gradient_single_runtime: 0.002228260040283203
2023-09-28 23:26:12,484 - utils - INFO -  epoch: 322, all client loss: [0.5824487805366516, 0.5026880502700806], all pred client disparities: [0.010958611965179443, 0.004941791296005249], all client disparities: [0.024637669324874878, 0.0008849650621414185], all client accs: [0.7481840252876282, 0.7737650871276855],  alpha_performance: tensor([0.5831, 0.4169], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:12,714 - utils - INFO - stage1_gradient_single_runtime: 0.0022530555725097656
2023-09-28 23:26:12,715 - utils - INFO -  epoch: 323, all client loss: [0.5827072262763977, 0.502431333065033], all pred client disparities: [0.010411113500595093, 0.004753544926643372], all client disparities: [0.024637669324874878, 0.00044643133878707886], all client accs: [0.7481840252876282, 0.7737962007522583],  alpha_performance: tensor([0.5864, 0.4136], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:12,945 - utils - INFO - stage1_gradient_single_runtime: 0.002071380615234375
2023-09-28 23:26:12,946 - utils - INFO -  epoch: 324, all client loss: [0.5829624533653259, 0.5021777153015137], all pred client disparities: [0.00985097885131836, 0.004573360085487366], all client disparities: [0.024637669324874878, 0.00044643133878707886], all client accs: [0.7481840252876282, 0.7738584280014038],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:13,182 - utils - INFO - stage1_gradient_single_runtime: 0.002274036407470703
2023-09-28 23:26:13,182 - utils - INFO -  epoch: 325, all client loss: [0.5816989541053772, 0.5021153688430786], all pred client disparities: [0.011313647031784058, 0.004988089203834534], all client disparities: [0.022826075553894043, 0.001031138002872467], all client accs: [0.7506053447723389, 0.7737339735031128],  alpha_performance: tensor([0.5807, 0.4193], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:13,420 - utils - INFO - stage1_gradient_single_runtime: 0.0026683807373046875
2023-09-28 23:26:13,421 - utils - INFO -  epoch: 326, all client loss: [0.58195561170578, 0.501860499382019], all pred client disparities: [0.010784715414047241, 0.004796802997589111], all client disparities: [0.022826075553894043, 0.00044643133878707886], all client accs: [0.7506053447723389, 0.7737962007522583],  alpha_performance: tensor([0.5840, 0.4160], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:13,653 - utils - INFO - stage1_gradient_single_runtime: 0.0022995471954345703
2023-09-28 23:26:13,653 - utils - INFO -  epoch: 327, all client loss: [0.5822089314460754, 0.5016087889671326], all pred client disparities: [0.01024356484413147, 0.00461365282535553], all client disparities: [0.022826075553894043, 0.00044643133878707886], all client accs: [0.7506053447723389, 0.7737962007522583],  alpha_performance: tensor([0.5873, 0.4127], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:13,885 - utils - INFO - stage1_gradient_single_runtime: 0.0020775794982910156
2023-09-28 23:26:13,886 - utils - INFO -  epoch: 328, all client loss: [0.5824591517448425, 0.5013601779937744], all pred client disparities: [0.009690165519714355, 0.004438385367393494], all client disparities: [0.022826075553894043, 0.00044643133878707886], all client accs: [0.7506053447723389, 0.77398282289505],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:14,172 - utils - INFO - stage1_gradient_single_runtime: 0.0023376941680908203
2023-09-28 23:26:14,173 - utils - INFO -  epoch: 329, all client loss: [0.5812016129493713, 0.5012999773025513], all pred client disparities: [0.011151373386383057, 0.004849523305892944], all client disparities: [0.022826075553894043, 0.001031138002872467], all client accs: [0.7506053447723389, 0.7737339735031128],  alpha_performance: tensor([0.5816, 0.4184], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:14,410 - utils - INFO - stage1_gradient_single_runtime: 0.0022428035736083984
2023-09-28 23:26:14,411 - utils - INFO -  epoch: 330, all client loss: [0.5814533829689026, 0.5010498762130737], all pred client disparities: [0.010628938674926758, 0.004663124680519104], all client disparities: [0.022826075553894043, 0.0005195215344429016], all client accs: [0.7506053447723389, 0.7737962007522583],  alpha_performance: tensor([0.5848, 0.4152], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:14,639 - utils - INFO - stage1_gradient_single_runtime: 0.0028204917907714844
2023-09-28 23:26:14,640 - utils - INFO -  epoch: 331, all client loss: [0.5817018747329712, 0.5008029341697693], all pred client disparities: [0.01009458303451538, 0.004484608769416809], all client disparities: [0.022826075553894043, 0.0005195215344429016], all client accs: [0.7506053447723389, 0.7739205956459045],  alpha_performance: tensor([0.5880, 0.4120], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:14,867 - utils - INFO - stage1_gradient_single_runtime: 0.002062082290649414
2023-09-28 23:26:14,868 - utils - INFO -  epoch: 332, all client loss: [0.581947386264801, 0.5005590319633484], all pred client disparities: [0.009548276662826538, 0.004313796758651733], all client disparities: [0.022826075553894043, 0.0005195215344429016], all client accs: [0.7506053447723389, 0.7740139365196228],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:15,101 - utils - INFO - stage1_gradient_single_runtime: 0.002264738082885742
2023-09-28 23:26:15,102 - utils - INFO -  epoch: 333, all client loss: [0.5806963443756104, 0.5005007386207581], all pred client disparities: [0.011006563901901245, 0.004721462726593018], all client disparities: [0.022826075553894043, 0.0005926117300987244], all client accs: [0.7506053447723389, 0.77370285987854],  alpha_performance: tensor([0.5823, 0.4177], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:15,330 - utils - INFO - stage1_gradient_single_runtime: 0.002321481704711914
2023-09-28 23:26:15,330 - utils - INFO -  epoch: 334, all client loss: [0.5809434652328491, 0.5002552270889282], all pred client disparities: [0.010490953922271729, 0.004539474844932556], all client disparities: [0.022826075553894043, 0.0005195215344429016], all client accs: [0.7506053447723389, 0.7739205956459045],  alpha_performance: tensor([0.5855, 0.4145], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:15,556 - utils - INFO - stage1_gradient_single_runtime: 0.0020363330841064453
2023-09-28 23:26:15,556 - utils - INFO -  epoch: 335, all client loss: [0.5811874866485596, 0.5000128149986267], all pred client disparities: [0.009963750839233398, 0.004365220665931702], all client disparities: [0.022826075553894043, 0.0005195215344429016], all client accs: [0.7506053447723389, 0.77398282289505],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:15,779 - utils - INFO - stage1_gradient_single_runtime: 0.0022735595703125
2023-09-28 23:26:15,780 - utils - INFO -  epoch: 336, all client loss: [0.5799546837806702, 0.49995356798171997], all pred client disparities: [0.011379420757293701, 0.004771709442138672], all client disparities: [0.021014481782913208, 0.000738784670829773], all client accs: [0.7506053447723389, 0.7736717462539673],  alpha_performance: tensor([0.5798, 0.4202], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:16,005 - utils - INFO - stage1_gradient_single_runtime: 0.0022401809692382812
2023-09-28 23:26:16,005 - utils - INFO -  epoch: 337, all client loss: [0.5802001953125, 0.4997096657752991], all pred client disparities: [0.010882139205932617, 0.00458642840385437], all client disparities: [0.022826075553894043, 0.0006656944751739502], all client accs: [0.7506053447723389, 0.7739205956459045],  alpha_performance: tensor([0.5830, 0.4170], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:16,232 - utils - INFO - stage1_gradient_single_runtime: 0.0022644996643066406
2023-09-28 23:26:16,233 - utils - INFO -  epoch: 338, all client loss: [0.5804427266120911, 0.49946877360343933], all pred client disparities: [0.010373622179031372, 0.004408881068229675], all client disparities: [0.022826075553894043, 0.0005926117300987244], all client accs: [0.7506053447723389, 0.77398282289505],  alpha_performance: tensor([0.5861, 0.4139], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:16,459 - utils - INFO - stage1_gradient_single_runtime: 0.0020904541015625
2023-09-28 23:26:16,460 - utils - INFO -  epoch: 339, all client loss: [0.580682098865509, 0.49923089146614075], all pred client disparities: [0.009853780269622803, 0.004238888621330261], all client disparities: [0.022826075553894043, 0.0005926117300987244], all client accs: [0.7506053447723389, 0.7740450501441956],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:16,544 - utils - INFO - valid: True, epoch: 339, loss: [0.6039758324623108, 0.4982425272464752], accuracy: [0.7016574740409851, 0.7773913145065308], mean_accuracy:0.7395243942737579,variance_accuracy:0.03786692023277283, disparity: [0.04545453190803528, 0.014579027891159058], mean_disparity:0.030016779899597168,variance_disparity:0.01543775200843811, pred_disparity: [0.035226792097091675, 0.001737162470817566]
2023-09-28 23:26:16,672 - utils - INFO - global_valid: True, epoch: 339,  global_loss: 0.4994180202484131, global_accuracy: 0.8080896271636953,  global_disparity:0.015438973903656006, global_pred_disparity: 0.0033183395862579346,
2023-09-28 23:26:16,897 - utils - INFO - stage1_gradient_single_runtime: 0.002235889434814453
2023-09-28 23:26:16,898 - utils - INFO -  epoch: 340, all client loss: [0.5794558525085449, 0.49917346239089966], all pred client disparities: [0.011265069246292114, 0.004641994833946228], all client disparities: [0.021014481782913208, 0.000738784670829773], all client accs: [0.7506053447723389, 0.7736717462539673],  alpha_performance: tensor([0.5803, 0.4197], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:17,124 - utils - INFO - stage1_gradient_single_runtime: 0.0022478103637695312
2023-09-28 23:26:17,125 - utils - INFO -  epoch: 341, all client loss: [0.5796969532966614, 0.4989340007305145], all pred client disparities: [0.010774880647659302, 0.004460930824279785], all client disparities: [0.021014481782913208, 0.0006656944751739502], all client accs: [0.7506053447723389, 0.7739517092704773],  alpha_performance: tensor([0.5835, 0.4165], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:17,351 - utils - INFO - stage1_gradient_single_runtime: 0.002268075942993164
2023-09-28 23:26:17,352 - utils - INFO -  epoch: 342, all client loss: [0.5799349546432495, 0.4986974895000458], all pred client disparities: [0.010273754596710205, 0.004287436604499817], all client disparities: [0.022826075553894043, 0.0005926117300987244], all client accs: [0.7506053447723389, 0.7740139365196228],  alpha_performance: tensor([0.5866, 0.4134], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:17,582 - utils - INFO - stage1_gradient_single_runtime: 0.002131938934326172
2023-09-28 23:26:17,583 - utils - INFO -  epoch: 343, all client loss: [0.5801700353622437, 0.4984639286994934], all pred client disparities: [0.009761512279510498, 0.004121303558349609], all client disparities: [0.022826075553894043, 0.0005926117300987244], all client accs: [0.7506053447723389, 0.7740450501441956],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:17,811 - utils - INFO - stage1_gradient_single_runtime: 0.0022513866424560547
2023-09-28 23:26:17,812 - utils - INFO -  epoch: 344, all client loss: [0.5789506435394287, 0.49840816855430603], all pred client disparities: [0.011167198419570923, 0.0045211464166641235], all client disparities: [0.021014481782913208, 0.000738784670829773], all client accs: [0.7506053447723389, 0.7739205956459045],  alpha_performance: tensor([0.5808, 0.4192], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:18,038 - utils - INFO - stage1_gradient_single_runtime: 0.002249479293823242
2023-09-28 23:26:18,039 - utils - INFO -  epoch: 345, all client loss: [0.5791874527931213, 0.49817290902137756], all pred client disparities: [0.010684370994567871, 0.004343941807746887], all client disparities: [0.021014481782913208, 0.000738784670829773], all client accs: [0.7506053447723389, 0.7739517092704773],  alpha_performance: tensor([0.5839, 0.4161], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:18,275 - utils - INFO - stage1_gradient_single_runtime: 0.0025894641876220703
2023-09-28 23:26:18,276 - utils - INFO -  epoch: 346, all client loss: [0.57942134141922, 0.4979405701160431], all pred client disparities: [0.010190844535827637, 0.004174143075942993], all client disparities: [0.021014481782913208, 0.0005926117300987244], all client accs: [0.7506053447723389, 0.7740139365196228],  alpha_performance: tensor([0.5869, 0.4131], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:18,530 - utils - INFO - stage1_gradient_single_runtime: 0.002166271209716797
2023-09-28 23:26:18,530 - utils - INFO -  epoch: 347, all client loss: [0.5796523094177246, 0.4977111220359802], all pred client disparities: [0.009686499834060669, 0.0040115416049957275], all client disparities: [0.021014481782913208, 0.0005926117300987244], all client accs: [0.7506053447723389, 0.7740450501441956],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:18,754 - utils - INFO - stage1_gradient_single_runtime: 0.002271890640258789
2023-09-28 23:26:18,755 - utils - INFO -  epoch: 348, all client loss: [0.578440248966217, 0.4976568818092346], all pred client disparities: [0.011085569858551025, 0.004408225417137146], all client disparities: [0.021014481782913208, 0.0009580552577972412], all client accs: [0.7506053447723389, 0.7740450501441956],  alpha_performance: tensor([0.5812, 0.4188], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:18,980 - utils - INFO - stage1_gradient_single_runtime: 0.002295970916748047
2023-09-28 23:26:18,981 - utils - INFO -  epoch: 349, all client loss: [0.5786730647087097, 0.4974256455898285], all pred client disparities: [0.010610312223434448, 0.004234522581100464], all client disparities: [0.021014481782913208, 0.000738784670829773], all client accs: [0.7506053447723389, 0.77398282289505],  alpha_performance: tensor([0.5842, 0.4158], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:19,209 - utils - INFO - stage1_gradient_single_runtime: 0.002285003662109375
2023-09-28 23:26:19,209 - utils - INFO -  epoch: 350, all client loss: [0.5789029598236084, 0.4971972703933716], all pred client disparities: [0.010124623775482178, 0.0040680766105651855], all client disparities: [0.021014481782913208, 0.0005926117300987244], all client accs: [0.7506053447723389, 0.7740139365196228],  alpha_performance: tensor([0.5872, 0.4128], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:19,384 - utils - INFO - stage1_gradient_single_runtime: 0.002080202102661133
2023-09-28 23:26:19,385 - utils - INFO -  epoch: 351, all client loss: [0.5791299939155579, 0.49697166681289673], all pred client disparities: [0.009628385305404663, 0.003908693790435791], all client disparities: [0.021014481782913208, 0.0015269666910171509], all client accs: [0.7506053447723389, 0.7773733735084534],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:19,612 - utils - INFO - stage1_gradient_single_runtime: 0.002245187759399414
2023-09-28 23:26:19,613 - utils - INFO -  epoch: 352, all client loss: [0.5779255628585815, 0.49691885709762573], all pred client disparities: [0.011019587516784668, 0.004302322864532471], all client disparities: [0.021014481782913208, 0.0009580552577972412], all client accs: [0.7506053447723389, 0.7740450501441956],  alpha_performance: tensor([0.5815, 0.4185], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:19,835 - utils - INFO - stage1_gradient_single_runtime: 0.0022623538970947266
2023-09-28 23:26:19,836 - utils - INFO -  epoch: 353, all client loss: [0.5781545042991638, 0.49669137597084045], all pred client disparities: [0.010552197694778442, 0.0041318535804748535], all client disparities: [0.021014481782913208, 0.0009580552577972412], all client accs: [0.7506053447723389, 0.7740761637687683],  alpha_performance: tensor([0.5845, 0.4155], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:20,064 - utils - INFO - stage1_gradient_single_runtime: 0.0022499561309814453
2023-09-28 23:26:20,065 - utils - INFO -  epoch: 354, all client loss: [0.5783805847167969, 0.49646681547164917], all pred client disparities: [0.010074496269226074, 0.003968477249145508], all client disparities: [0.021014481782913208, 0.0014538764953613281], all client accs: [0.7506053447723389, 0.7773422598838806],  alpha_performance: tensor([0.5874, 0.4126], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:20,295 - utils - INFO - stage1_gradient_single_runtime: 0.0020873546600341797
2023-09-28 23:26:20,296 - utils - INFO -  epoch: 355, all client loss: [0.5786038637161255, 0.4962449371814728], all pred client disparities: [0.009586602449417114, 0.0038120299577713013], all client disparities: [0.021014481782913208, 0.0015269666910171509], all client accs: [0.7506053447723389, 0.7773733735084534],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:20,523 - utils - INFO - stage1_gradient_single_runtime: 0.002242565155029297
2023-09-28 23:26:20,524 - utils - INFO -  epoch: 356, all client loss: [0.5774074196815491, 0.4961933493614197], all pred client disparities: [0.010969012975692749, 0.004202693700790405], all client disparities: [0.021014481782913208, 0.0009580552577972412], all client accs: [0.7506053447723389, 0.7740761637687683],  alpha_performance: tensor([0.5817, 0.4183], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:20,751 - utils - INFO - stage1_gradient_single_runtime: 0.0022878646850585938
2023-09-28 23:26:20,752 - utils - INFO -  epoch: 357, all client loss: [0.5776326656341553, 0.49596959352493286], all pred client disparities: [0.010509580373764038, 0.00403517484664917], all client disparities: [0.021014481782913208, 0.0009580552577972412], all client accs: [0.7506053447723389, 0.7741072773933411],  alpha_performance: tensor([0.5847, 0.4153], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:20,986 - utils - INFO - stage1_gradient_single_runtime: 0.0023114681243896484
2023-09-28 23:26:20,986 - utils - INFO -  epoch: 358, all client loss: [0.5778551697731018, 0.4957485795021057], all pred client disparities: [0.010040134191513062, 0.0038746148347854614], all client disparities: [0.021014481782913208, 0.0012346059083938599], all client accs: [0.7506053447723389, 0.7774356007575989],  alpha_performance: tensor([0.5876, 0.4124], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:21,223 - utils - INFO - stage1_gradient_single_runtime: 0.0021343231201171875
2023-09-28 23:26:21,224 - utils - INFO -  epoch: 359, all client loss: [0.5780749320983887, 0.49553027749061584], all pred client disparities: [0.00956067442893982, 0.003720805048942566], all client disparities: [0.021014481782913208, 0.0015269666910171509], all client accs: [0.7506053447723389, 0.7774356007575989],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:21,310 - utils - INFO - valid: True, epoch: 359, loss: [0.6022170782089233, 0.49453333020210266], accuracy: [0.6961326003074646, 0.7773913145065308], mean_accuracy:0.7367619574069977,variance_accuracy:0.04062935709953308, disparity: [0.04545453190803528, 0.014431104063987732], mean_disparity:0.029942817986011505,variance_disparity:0.015511713922023773, pred_disparity: [0.035573095083236694, 0.0021406561136245728]
2023-09-28 23:26:21,439 - utils - INFO - global_valid: True, epoch: 359,  global_loss: 0.49573051929473877, global_accuracy: 0.8083023610449847,  global_disparity:0.01529569923877716, global_pred_disparity: 0.003719925880432129,
2023-09-28 23:26:21,668 - utils - INFO - stage1_gradient_single_runtime: 0.0027549266815185547
2023-09-28 23:26:21,669 - utils - INFO -  epoch: 360, all client loss: [0.5768866539001465, 0.4954798221588135], all pred client disparities: [0.010933458805084229, 0.004108652472496033], all client disparities: [0.021014481782913208, 0.0009580552577972412], all client accs: [0.7506053447723389, 0.7740761637687683],  alpha_performance: tensor([0.5819, 0.4181], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:21,897 - utils - INFO - stage1_gradient_single_runtime: 0.0022842884063720703
2023-09-28 23:26:21,898 - utils - INFO -  epoch: 361, all client loss: [0.5771083831787109, 0.495259553194046], all pred client disparities: [0.010482043027877808, 0.003943800926208496], all client disparities: [0.021014481782913208, 0.001161523163318634], all client accs: [0.7506053447723389, 0.7773733735084534],  alpha_performance: tensor([0.5847, 0.4153], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:22,127 - utils - INFO - stage1_gradient_single_runtime: 0.0022559165954589844
2023-09-28 23:26:22,128 - utils - INFO -  epoch: 362, all client loss: [0.5773273706436157, 0.49504202604293823], all pred client disparities: [0.010021060705184937, 0.0037858039140701294], all client disparities: [0.021014481782913208, 0.0012346059083938599], all client accs: [0.7506053447723389, 0.7774356007575989],  alpha_performance: tensor([0.5876, 0.4124], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:22,355 - utils - INFO - stage1_gradient_single_runtime: 0.0020835399627685547
2023-09-28 23:26:22,356 - utils - INFO -  epoch: 363, all client loss: [0.5775437355041504, 0.49482715129852295], all pred client disparities: [0.009550213813781738, 0.003634423017501831], all client disparities: [0.021014481782913208, 0.0014538764953613281], all client accs: [0.7506053447723389, 0.7774356007575989],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:22,578 - utils - INFO - stage1_gradient_single_runtime: 0.002302408218383789
2023-09-28 23:26:22,578 - utils - INFO -  epoch: 364, all client loss: [0.5763638615608215, 0.49477776885032654], all pred client disparities: [0.010912328958511353, 0.004019543528556824], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7774667143821716],  alpha_performance: tensor([0.5819, 0.4181], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:22,800 - utils - INFO - stage1_gradient_single_runtime: 0.0022575855255126953
2023-09-28 23:26:22,801 - utils - INFO -  epoch: 365, all client loss: [0.5765822529792786, 0.4945608675479889], all pred client disparities: [0.010469257831573486, 0.003857135772705078], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7774667143821716],  alpha_performance: tensor([0.5847, 0.4153], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:23,024 - utils - INFO - stage1_gradient_single_runtime: 0.002265453338623047
2023-09-28 23:26:23,025 - utils - INFO -  epoch: 366, all client loss: [0.5767979025840759, 0.49434664845466614], all pred client disparities: [0.010016769170761108, 0.003701448440551758], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7775600552558899],  alpha_performance: tensor([0.5876, 0.4124], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:23,243 - utils - INFO - stage1_gradient_single_runtime: 0.0020723342895507812
2023-09-28 23:26:23,244 - utils - INFO -  epoch: 367, all client loss: [0.5770108699798584, 0.4941350221633911], all pred client disparities: [0.009554624557495117, 0.0035523027181625366], all client disparities: [0.021014481782913208, 0.0012346059083938599], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:23,460 - utils - INFO - stage1_gradient_single_runtime: 0.0023488998413085938
2023-09-28 23:26:23,461 - utils - INFO -  epoch: 368, all client loss: [0.5758396983146667, 0.4940866529941559], all pred client disparities: [0.010905236005783081, 0.003934785723686218], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7774667143821716],  alpha_performance: tensor([0.5819, 0.4181], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:23,697 - utils - INFO - stage1_gradient_single_runtime: 0.0022966861724853516
2023-09-28 23:26:23,698 - utils - INFO -  epoch: 369, all client loss: [0.5760547518730164, 0.4938729405403137], all pred client disparities: [0.010470598936080933, 0.0037746280431747437], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7774667143821716],  alpha_performance: tensor([0.5847, 0.4153], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:23,924 - utils - INFO - stage1_gradient_single_runtime: 0.0022661685943603516
2023-09-28 23:26:23,924 - utils - INFO -  epoch: 370, all client loss: [0.5762673020362854, 0.49366194009780884], all pred client disparities: [0.010026723146438599, 0.003621041774749756], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5875, 0.4125], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:24,147 - utils - INFO - stage1_gradient_single_runtime: 0.002091646194458008
2023-09-28 23:26:24,148 - utils - INFO -  epoch: 371, all client loss: [0.57647705078125, 0.4934535026550293], all pred client disparities: [0.009573489427566528, 0.0034738928079605103], all client disparities: [0.021014481782913208, 0.0033959895372390747], all client accs: [0.7506053447723389, 0.7776222229003906],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:24,415 - utils - INFO - stage1_gradient_single_runtime: 0.00225830078125
2023-09-28 23:26:24,415 - utils - INFO -  epoch: 372, all client loss: [0.5753147006034851, 0.4934059977531433], all pred client disparities: [0.01091185212135315, 0.0038538575172424316], all client disparities: [0.021014481782913208, 0.004763834178447723], all client accs: [0.7506053447723389, 0.7774978280067444],  alpha_performance: tensor([0.5817, 0.4183], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:24,641 - utils - INFO - stage1_gradient_single_runtime: 0.0022668838500976562
2023-09-28 23:26:24,642 - utils - INFO -  epoch: 373, all client loss: [0.5755266547203064, 0.49319544434547424], all pred client disparities: [0.010485649108886719, 0.003695756196975708], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5845, 0.4155], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:24,867 - utils - INFO - stage1_gradient_single_runtime: 0.002262115478515625
2023-09-28 23:26:24,868 - utils - INFO -  epoch: 374, all client loss: [0.5757359862327576, 0.4929874837398529], all pred client disparities: [0.010050535202026367, 0.003544136881828308], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5872, 0.4128], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:25,096 - utils - INFO - stage1_gradient_single_runtime: 0.002086639404296875
2023-09-28 23:26:25,096 - utils - INFO -  epoch: 375, all client loss: [0.5759427547454834, 0.4927820861339569], all pred client disparities: [0.009606361389160156, 0.003398790955543518], all client disparities: [0.021014481782913208, 0.0033959895372390747], all client accs: [0.7506053447723389, 0.7776222229003906],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:25,321 - utils - INFO - stage1_gradient_single_runtime: 0.002376079559326172
2023-09-28 23:26:25,322 - utils - INFO -  epoch: 376, all client loss: [0.5747894644737244, 0.4927353858947754], all pred client disparities: [0.01093149185180664, 0.0037763267755508423], all client disparities: [0.021014481782913208, 0.004763834178447723], all client accs: [0.7506053447723389, 0.7774667143821716],  alpha_performance: tensor([0.5815, 0.4185], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:25,557 - utils - INFO - stage1_gradient_single_runtime: 0.0031604766845703125
2023-09-28 23:26:25,558 - utils - INFO -  epoch: 377, all client loss: [0.5749983191490173, 0.49252790212631226], all pred client disparities: [0.010513991117477417, 0.0036200881004333496], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7774978280067444],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:25,825 - utils - INFO - stage1_gradient_single_runtime: 0.0036859512329101562
2023-09-28 23:26:25,826 - utils - INFO -  epoch: 378, all client loss: [0.5752046704292297, 0.4923229515552521], all pred client disparities: [0.010087698698043823, 0.003470182418823242], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7774978280067444],  alpha_performance: tensor([0.5870, 0.4130], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:26,053 - utils - INFO - stage1_gradient_single_runtime: 0.0021059513092041016
2023-09-28 23:26:26,054 - utils - INFO -  epoch: 379, all client loss: [0.5754084587097168, 0.4921204745769501], all pred client disparities: [0.009652554988861084, 0.003326505422592163], all client disparities: [0.021014481782913208, 0.0032602399587631226], all client accs: [0.7506053447723389, 0.7776844501495361],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:26,186 - utils - INFO - valid: True, epoch: 379, loss: [0.600429117679596, 0.4911141097545624], accuracy: [0.6961326003074646, 0.7800000309944153], mean_accuracy:0.7380663156509399,variance_accuracy:0.04193371534347534, disparity: [0.04545453190803528, 0.012829914689064026], mean_disparity:0.029142223298549652,variance_disparity:0.016312308609485626, pred_disparity: [0.0354999303817749, 0.0024029910564422607]
2023-09-28 23:26:26,318 - utils - INFO - global_valid: True, epoch: 379,  global_loss: 0.4923294186592102, global_accuracy: 0.808638105878075,  global_disparity:0.013776108622550964, global_pred_disparity: 0.003996610641479492,
2023-09-28 23:26:26,538 - utils - INFO - stage1_gradient_single_runtime: 0.0022859573364257812
2023-09-28 23:26:26,538 - utils - INFO -  epoch: 380, all client loss: [0.5742641687393188, 0.49207454919815063], all pred client disparities: [0.010963886976242065, 0.003701731562614441], all client disparities: [0.021014481782913208, 0.004617653787136078], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5812, 0.4188], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:26,761 - utils - INFO - stage1_gradient_single_runtime: 0.002277851104736328
2023-09-28 23:26:26,761 - utils - INFO -  epoch: 381, all client loss: [0.5744701623916626, 0.4918699562549591], all pred client disparities: [0.010555088520050049, 0.003547176718711853], all client disparities: [0.021014481782913208, 0.004763834178447723], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5839, 0.4161], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:26,985 - utils - INFO - stage1_gradient_single_runtime: 0.0022547245025634766
2023-09-28 23:26:26,986 - utils - INFO -  epoch: 382, all client loss: [0.574673593044281, 0.4916679263114929], all pred client disparities: [0.0101376473903656, 0.0033988654613494873], all client disparities: [0.021014481782913208, 0.003322906792163849], all client accs: [0.7506053447723389, 0.7774978280067444],  alpha_performance: tensor([0.5866, 0.4134], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:27,212 - utils - INFO - stage1_gradient_single_runtime: 0.0020809173583984375
2023-09-28 23:26:27,213 - utils - INFO -  epoch: 383, all client loss: [0.5748744606971741, 0.4914683699607849], all pred client disparities: [0.009711682796478271, 0.003256678581237793], all client disparities: [0.021014481782913208, 0.0032602399587631226], all client accs: [0.7506053447723389, 0.7776533365249634],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:27,435 - utils - INFO - stage1_gradient_single_runtime: 0.0022459030151367188
2023-09-28 23:26:27,436 - utils - INFO -  epoch: 384, all client loss: [0.5737394690513611, 0.4914231300354004], all pred client disparities: [0.011008530855178833, 0.0036296844482421875], all client disparities: [0.021014481782913208, 0.004617653787136078], all client accs: [0.7506053447723389, 0.7775600552558899],  alpha_performance: tensor([0.5809, 0.4191], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:27,663 - utils - INFO - stage1_gradient_single_runtime: 0.002657651901245117
2023-09-28 23:26:27,664 - utils - INFO -  epoch: 385, all client loss: [0.5739426016807556, 0.4912213981151581], all pred client disparities: [0.010608404874801636, 0.003476664423942566], all client disparities: [0.021014481782913208, 0.004763834178447723], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5835, 0.4165], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:27,914 - utils - INFO - stage1_gradient_single_runtime: 0.002507448196411133
2023-09-28 23:26:27,915 - utils - INFO -  epoch: 386, all client loss: [0.5741431713104248, 0.49102216958999634], all pred client disparities: [0.01020011305809021, 0.0033297836780548096], all client disparities: [0.021014481782913208, 0.004910007119178772], all client accs: [0.7506053447723389, 0.7774978280067444],  alpha_performance: tensor([0.5861, 0.4139], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:28,152 - utils - INFO - stage1_gradient_single_runtime: 0.0021224021911621094
2023-09-28 23:26:28,153 - utils - INFO -  epoch: 387, all client loss: [0.5743412375450134, 0.49082544445991516], all pred client disparities: [0.009783297777175903, 0.0031889379024505615], all client disparities: [0.021014481782913208, 0.0032602399587631226], all client accs: [0.7506053447723389, 0.7776533365249634],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:28,381 - utils - INFO - stage1_gradient_single_runtime: 0.002270221710205078
2023-09-28 23:26:28,382 - utils - INFO -  epoch: 388, all client loss: [0.5732156038284302, 0.4907808005809784], all pred client disparities: [0.011064887046813965, 0.003559812903404236], all client disparities: [0.021014481782913208, 0.004617653787136078], all client accs: [0.7506053447723389, 0.7775600552558899],  alpha_performance: tensor([0.5804, 0.4196], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:28,608 - utils - INFO - stage1_gradient_single_runtime: 0.002305746078491211
2023-09-28 23:26:28,609 - utils - INFO -  epoch: 389, all client loss: [0.5734158754348755, 0.49058184027671814], all pred client disparities: [0.0106736421585083, 0.0034081637859344482], all client disparities: [0.021014481782913208, 0.004910007119178772], all client accs: [0.7506053447723389, 0.7774667143821716],  alpha_performance: tensor([0.5830, 0.4170], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:28,836 - utils - INFO - stage1_gradient_single_runtime: 0.002560138702392578
2023-09-28 23:26:28,837 - utils - INFO -  epoch: 390, all client loss: [0.573613703250885, 0.49038535356521606], all pred client disparities: [0.010274261236190796, 0.0032625794410705566], all client disparities: [0.021014481782913208, 0.004701167345046997], all client accs: [0.7506053447723389, 0.7776533365249634],  alpha_performance: tensor([0.5856, 0.4144], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:29,069 - utils - INFO - stage1_gradient_single_runtime: 0.002243518829345703
2023-09-28 23:26:29,069 - utils - INFO -  epoch: 391, all client loss: [0.573809027671814, 0.4901913106441498], all pred client disparities: [0.009866833686828613, 0.003122955560684204], all client disparities: [0.021014481782913208, 0.003698773682117462], all client accs: [0.7506053447723389, 0.7788975834846497],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:29,292 - utils - INFO - stage1_gradient_single_runtime: 0.00228118896484375
2023-09-28 23:26:29,293 - utils - INFO -  epoch: 392, all client loss: [0.5726928114891052, 0.49014726281166077], all pred client disparities: [0.011132538318634033, 0.0034917593002319336], all client disparities: [0.021014481782913208, 0.004617653787136078], all client accs: [0.7506053447723389, 0.7775600552558899],  alpha_performance: tensor([0.5799, 0.4201], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:29,525 - utils - INFO - stage1_gradient_single_runtime: 0.0022516250610351562
2023-09-28 23:26:29,526 - utils - INFO -  epoch: 393, all client loss: [0.5728904008865356, 0.4899510145187378], all pred client disparities: [0.010750174522399902, 0.003341376781463623], all client disparities: [0.021014481782913208, 0.004763834178447723], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5824, 0.4176], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:29,748 - utils - INFO - stage1_gradient_single_runtime: 0.0022611618041992188
2023-09-28 23:26:29,749 - utils - INFO -  epoch: 394, all client loss: [0.5730854868888855, 0.489757239818573], all pred client disparities: [0.010359913110733032, 0.0031969845294952393], all client disparities: [0.021014481782913208, 0.004701167345046997], all client accs: [0.7506053447723389, 0.7776222229003906],  alpha_performance: tensor([0.5850, 0.4150], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:29,969 - utils - INFO - stage1_gradient_single_runtime: 0.0021147727966308594
2023-09-28 23:26:29,969 - utils - INFO -  epoch: 395, all client loss: [0.5732781291007996, 0.4895658493041992], all pred client disparities: [0.009961724281311035, 0.00305841863155365], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7789286971092224],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:30,192 - utils - INFO - stage1_gradient_single_runtime: 0.002241373062133789
2023-09-28 23:26:30,193 - utils - INFO -  epoch: 396, all client loss: [0.5721715092658997, 0.4895223379135132], all pred client disparities: [0.011211037635803223, 0.0034252703189849854], all client disparities: [0.021014481782913208, 0.004763834178447723], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5793, 0.4207], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:30,415 - utils - INFO - stage1_gradient_single_runtime: 0.0022749900817871094
2023-09-28 23:26:30,416 - utils - INFO -  epoch: 397, all client loss: [0.5723664164543152, 0.4893287420272827], all pred client disparities: [0.010837525129318237, 0.0032759904861450195], all client disparities: [0.021014481782913208, 0.004763834178447723], all client accs: [0.7506053447723389, 0.7775289416313171],  alpha_performance: tensor([0.5818, 0.4182], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:30,643 - utils - INFO - stage1_gradient_single_runtime: 0.002291440963745117
2023-09-28 23:26:30,644 - utils - INFO -  epoch: 398, all client loss: [0.5725588798522949, 0.48913758993148804], all pred client disparities: [0.010456383228302002, 0.0031326264142990112], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7788664698600769],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:30,870 - utils - INFO - stage1_gradient_single_runtime: 0.0022568702697753906
2023-09-28 23:26:30,870 - utils - INFO -  epoch: 399, all client loss: [0.5727489590644836, 0.4889488220214844], all pred client disparities: [0.010067492723464966, 0.002995029091835022], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7788975834846497],  alpha_performance: tensor([0.5868, 0.4132], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:30,955 - utils - INFO - valid: True, epoch: 399, loss: [0.5995649099349976, 0.48778045177459717], accuracy: [0.6961326003074646, 0.7806832194328308], mean_accuracy:0.7384079098701477,variance_accuracy:0.042275309562683105, disparity: [0.04545453190803528, 0.013286717236042023], mean_disparity:0.02937062457203865,variance_disparity:0.016083907335996628, pred_disparity: [0.03680655360221863, 0.003144368529319763]
2023-09-28 23:26:31,084 - utils - INFO - global_valid: True, epoch: 399,  global_loss: 0.48902323842048645, global_accuracy: 0.8085573949697031,  global_disparity:0.014234080910682678, global_pred_disparity: 0.004688084125518799,
2023-09-28 23:26:31,304 - utils - INFO - stage1_gradient_single_runtime: 0.002340555191040039
2023-09-28 23:26:31,305 - utils - INFO -  epoch: 400, all client loss: [0.5729367136955261, 0.4887622892856598], all pred client disparities: [0.009670853614807129, 0.0028630346059799194], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789598107337952],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:31,531 - utils - INFO - stage1_gradient_single_runtime: 0.0022630691528320312
2023-09-28 23:26:31,532 - utils - INFO -  epoch: 401, all client loss: [0.5718308091163635, 0.4887210428714752], all pred client disparities: [0.010932385921478271, 0.003225475549697876], all client disparities: [0.021014481782913208, 0.0045549869537353516], all client accs: [0.7506053447723389, 0.7776844501495361],  alpha_performance: tensor([0.5811, 0.4189], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:31,755 - utils - INFO - stage1_gradient_single_runtime: 0.002239227294921875
2023-09-28 23:26:31,757 - utils - INFO -  epoch: 402, all client loss: [0.572020947933197, 0.488532155752182], all pred client disparities: [0.010560333728790283, 0.0030827373266220093], all client disparities: [0.021014481782913208, 0.004993520677089691], all client accs: [0.7506053447723389, 0.7789286971092224],  alpha_performance: tensor([0.5836, 0.4164], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:31,986 - utils - INFO - stage1_gradient_single_runtime: 0.002275705337524414
2023-09-28 23:26:31,988 - utils - INFO -  epoch: 403, all client loss: [0.5722087025642395, 0.4883457124233246], all pred client disparities: [0.010180860757827759, 0.0029457062482833862], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7788975834846497],  alpha_performance: tensor([0.5861, 0.4139], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:32,222 - utils - INFO - stage1_gradient_single_runtime: 0.002241373062133789
2023-09-28 23:26:32,223 - utils - INFO -  epoch: 404, all client loss: [0.5723941326141357, 0.48816150426864624], all pred client disparities: [0.009793668985366821, 0.0028142035007476807], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789286971092224],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:32,451 - utils - INFO - stage1_gradient_single_runtime: 0.0022797584533691406
2023-09-28 23:26:32,453 - utils - INFO -  epoch: 405, all client loss: [0.5712984800338745, 0.48812049627304077], all pred client disparities: [0.011036902666091919, 0.0031750351190567017], all client disparities: [0.021014481782913208, 0.004993520677089691], all client accs: [0.7506053447723389, 0.7789286971092224],  alpha_performance: tensor([0.5804, 0.4196], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:32,687 - utils - INFO - stage1_gradient_single_runtime: 0.002254009246826172
2023-09-28 23:26:32,688 - utils - INFO -  epoch: 406, all client loss: [0.5714862942695618, 0.4879339635372162], all pred client disparities: [0.010673969984054565, 0.0030328184366226196], all client disparities: [0.021014481782913208, 0.005066610872745514], all client accs: [0.7506053447723389, 0.7789598107337952],  alpha_performance: tensor([0.5828, 0.4172], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:32,915 - utils - INFO - stage1_gradient_single_runtime: 0.002572774887084961
2023-09-28 23:26:32,916 - utils - INFO -  epoch: 407, all client loss: [0.5716717839241028, 0.48774975538253784], all pred client disparities: [0.010303765535354614, 0.0028962641954421997], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789286971092224],  alpha_performance: tensor([0.5852, 0.4148], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:33,146 - utils - INFO - stage1_gradient_single_runtime: 0.0020973682403564453
2023-09-28 23:26:33,146 - utils - INFO -  epoch: 408, all client loss: [0.5718549489974976, 0.48756781220436096], all pred client disparities: [0.00992622971534729, 0.002765178680419922], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:33,377 - utils - INFO - stage1_gradient_single_runtime: 0.002566099166870117
2023-09-28 23:26:33,378 - utils - INFO -  epoch: 409, all client loss: [0.5707694888114929, 0.48752710223197937], all pred client disparities: [0.011150538921356201, 0.003124430775642395], all client disparities: [0.021014481782913208, 0.005066610872745514], all client accs: [0.7506053447723389, 0.7789598107337952],  alpha_performance: tensor([0.5795, 0.4205], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:33,607 - utils - INFO - stage1_gradient_single_runtime: 0.0023889541625976562
2023-09-28 23:26:33,609 - utils - INFO -  epoch: 410, all client loss: [0.5709550380706787, 0.48734283447265625], all pred client disparities: [0.010796785354614258, 0.00298270583152771], all client disparities: [0.021014481782913208, 0.005066610872745514], all client accs: [0.7506053447723389, 0.7789598107337952],  alpha_performance: tensor([0.5819, 0.4181], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:33,787 - utils - INFO - stage1_gradient_single_runtime: 0.002257108688354492
2023-09-28 23:26:33,787 - utils - INFO -  epoch: 411, all client loss: [0.5711382627487183, 0.48716089129447937], all pred client disparities: [0.010435909032821655, 0.002846553921699524], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:34,008 - utils - INFO - stage1_gradient_single_runtime: 0.0022535324096679688
2023-09-28 23:26:34,009 - utils - INFO -  epoch: 412, all client loss: [0.5713192224502563, 0.48698118329048157], all pred client disparities: [0.010067880153656006, 0.0027158111333847046], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([0.5867, 0.4133], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:34,232 - utils - INFO - stage1_gradient_single_runtime: 0.0020804405212402344
2023-09-28 23:26:34,232 - utils - INFO -  epoch: 413, all client loss: [0.5714979767799377, 0.48680365085601807], all pred client disparities: [0.009692579507827759, 0.0025903433561325073], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:34,454 - utils - INFO - stage1_gradient_single_runtime: 0.0022547245025634766
2023-09-28 23:26:34,455 - utils - INFO -  epoch: 414, all client loss: [0.5704143047332764, 0.4867648184299469], all pred client disparities: [0.01092541217803955, 0.0029455721378326416], all client disparities: [0.021014481782913208, 0.005066610872745514], all client accs: [0.7506053447723389, 0.7789598107337952],  alpha_performance: tensor([0.5811, 0.4189], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:34,680 - utils - INFO - stage1_gradient_single_runtime: 0.002293825149536133
2023-09-28 23:26:34,681 - utils - INFO -  epoch: 415, all client loss: [0.5705955028533936, 0.48658487200737], all pred client disparities: [0.01057383418083191, 0.0028094500303268433], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7790531516075134],  alpha_performance: tensor([0.5834, 0.4166], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:34,904 - utils - INFO - stage1_gradient_single_runtime: 0.0022792816162109375
2023-09-28 23:26:34,905 - utils - INFO -  epoch: 416, all client loss: [0.570774495601654, 0.4864071011543274], all pred client disparities: [0.010215282440185547, 0.0026787221431732178], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([0.5858, 0.4142], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:35,133 - utils - INFO - stage1_gradient_single_runtime: 0.002106189727783203
2023-09-28 23:26:35,134 - utils - INFO -  epoch: 417, all client loss: [0.5709512829780579, 0.48623156547546387], all pred client disparities: [0.009849756956100464, 0.002553209662437439], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:35,359 - utils - INFO - stage1_gradient_single_runtime: 0.0022516250610351562
2023-09-28 23:26:35,359 - utils - INFO -  epoch: 418, all client loss: [0.5698784589767456, 0.48619288206100464], all pred client disparities: [0.01106211543083191, 0.00290718674659729], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7789286971092224],  alpha_performance: tensor([0.5801, 0.4199], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:35,586 - utils - INFO - stage1_gradient_single_runtime: 0.0022542476654052734
2023-09-28 23:26:35,586 - utils - INFO -  epoch: 419, all client loss: [0.5700576901435852, 0.48601487278938293], all pred client disparities: [0.01071980595588684, 0.0027710795402526855], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7790531516075134],  alpha_performance: tensor([0.5824, 0.4176], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:35,672 - utils - INFO - valid: True, epoch: 419, loss: [0.5977305769920349, 0.4848456382751465], accuracy: [0.6961326003074646, 0.7806832194328308], mean_accuracy:0.7384079098701477,variance_accuracy:0.042275309562683105, disparity: [0.04545453190803528, 0.012842930853366852], mean_disparity:0.029148731380701065,variance_disparity:0.016305800527334213, pred_disparity: [0.0359342098236084, 0.003190204268321395]
2023-09-28 23:26:35,796 - utils - INFO - global_valid: True, epoch: 419,  global_loss: 0.48610061407089233, global_accuracy: 0.8092177607490307,  global_disparity:0.01380428671836853, global_pred_disparity: 0.004779741168022156,
2023-09-28 23:26:36,014 - utils - INFO - stage1_gradient_single_runtime: 0.0022776126861572266
2023-09-28 23:26:36,015 - utils - INFO -  epoch: 420, all client loss: [0.5702345967292786, 0.4858391284942627], all pred client disparities: [0.01037067174911499, 0.0026403069496154785], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7790531516075134],  alpha_performance: tensor([0.5848, 0.4152], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:36,236 - utils - INFO - stage1_gradient_single_runtime: 0.002252817153930664
2023-09-28 23:26:36,237 - utils - INFO -  epoch: 421, all client loss: [0.5704094171524048, 0.48566552996635437], all pred client disparities: [0.010014772415161133, 0.0025146901607513428], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([0.5871, 0.4129], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:36,466 - utils - INFO - stage1_gradient_single_runtime: 0.0020859241485595703
2023-09-28 23:26:36,467 - utils - INFO -  epoch: 422, all client loss: [0.5705820918083191, 0.4854940176010132], all pred client disparities: [0.009651929140090942, 0.002394154667854309], all client disparities: [0.021014481782913208, 0.012250155210494995], all client accs: [0.7506053447723389, 0.7763468623161316],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:36,688 - utils - INFO - stage1_gradient_single_runtime: 0.002254962921142578
2023-09-28 23:26:36,689 - utils - INFO -  epoch: 423, all client loss: [0.5695118308067322, 0.4854569733142853], all pred client disparities: [0.010870397090911865, 0.0027443766593933105], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7790531516075134],  alpha_performance: tensor([0.5814, 0.4186], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:36,912 - utils - INFO - stage1_gradient_single_runtime: 0.002261638641357422
2023-09-28 23:26:36,913 - utils - INFO -  epoch: 424, all client loss: [0.569687008857727, 0.4852829575538635], all pred client disparities: [0.010530680418014526, 0.002613261342048645], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7790531516075134],  alpha_performance: tensor([0.5837, 0.4163], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:37,134 - utils - INFO - stage1_gradient_single_runtime: 0.0022749900817871094
2023-09-28 23:26:37,135 - utils - INFO -  epoch: 425, all client loss: [0.5698601007461548, 0.4851111173629761], all pred client disparities: [0.010184377431869507, 0.0024872422218322754], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([0.5860, 0.4140], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:37,357 - utils - INFO - stage1_gradient_single_runtime: 0.0021071434020996094
2023-09-28 23:26:37,357 - utils - INFO -  epoch: 426, all client loss: [0.5700310468673706, 0.48494139313697815], all pred client disparities: [0.009831368923187256, 0.0023662596940994263], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:37,581 - utils - INFO - stage1_gradient_single_runtime: 0.0022575855255126953
2023-09-28 23:26:37,582 - utils - INFO -  epoch: 427, all client loss: [0.5689719915390015, 0.48490428924560547], all pred client disparities: [0.011028021574020386, 0.0027155429124832153], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7790531516075134],  alpha_performance: tensor([0.5804, 0.4196], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:37,802 - utils - INFO - stage1_gradient_single_runtime: 0.0022416114807128906
2023-09-28 23:26:37,803 - utils - INFO -  epoch: 428, all client loss: [0.5691453814506531, 0.48473209142684937], all pred client disparities: [0.010697603225708008, 0.0025840550661087036], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7790531516075134],  alpha_performance: tensor([0.5826, 0.4174], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:38,026 - utils - INFO - stage1_gradient_single_runtime: 0.002285003662109375
2023-09-28 23:26:38,027 - utils - INFO -  epoch: 429, all client loss: [0.5693166255950928, 0.4845620095729828], all pred client disparities: [0.010360807180404663, 0.002457648515701294], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7506053447723389, 0.7790531516075134],  alpha_performance: tensor([0.5849, 0.4151], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:38,252 - utils - INFO - stage1_gradient_single_runtime: 0.002240896224975586
2023-09-28 23:26:38,252 - utils - INFO -  epoch: 430, all client loss: [0.5694857835769653, 0.48439401388168335], all pred client disparities: [0.010017544031143188, 0.0023362338542938232], all client disparities: [0.021014481782913208, 0.005285874009132385], all client accs: [0.7506053447723389, 0.7789909243583679],  alpha_performance: tensor([0.5871, 0.4129], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:38,474 - utils - INFO - stage1_gradient_single_runtime: 0.002062559127807617
2023-09-28 23:26:38,475 - utils - INFO -  epoch: 431, all client loss: [0.5696529150009155, 0.4842280447483063], all pred client disparities: [0.009667694568634033, 0.002219647169113159], all client disparities: [0.021014481782913208, 0.011675871908664703], all client accs: [0.7506053447723389, 0.7762846946716309],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:38,698 - utils - INFO - stage1_gradient_single_runtime: 0.002232074737548828
2023-09-28 23:26:38,699 - utils - INFO -  epoch: 432, all client loss: [0.5685970187187195, 0.4841923713684082], all pred client disparities: [0.010868072509765625, 0.002565443515777588], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7790220379829407],  alpha_performance: tensor([0.5815, 0.4185], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:38,923 - utils - INFO - stage1_gradient_single_runtime: 0.0022301673889160156
2023-09-28 23:26:38,924 - utils - INFO -  epoch: 433, all client loss: [0.5687667727470398, 0.4840238690376282], all pred client disparities: [0.010540783405303955, 0.0024383515119552612], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7790531516075134],  alpha_performance: tensor([0.5837, 0.4163], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:39,146 - utils - INFO - stage1_gradient_single_runtime: 0.002245187759399414
2023-09-28 23:26:39,147 - utils - INFO -  epoch: 434, all client loss: [0.5689343214035034, 0.4838574230670929], all pred client disparities: [0.01020708680152893, 0.002316221594810486], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7790531516075134],  alpha_performance: tensor([0.5859, 0.4141], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:39,372 - utils - INFO - stage1_gradient_single_runtime: 0.0020818710327148438
2023-09-28 23:26:39,373 - utils - INFO -  epoch: 435, all client loss: [0.5690998435020447, 0.48369303345680237], all pred client disparities: [0.009867042303085327, 0.002198919653892517], all client disparities: [0.021014481782913208, 0.01160278171300888], all client accs: [0.7506053447723389, 0.7763158082962036],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:39,602 - utils - INFO - stage1_gradient_single_runtime: 0.002218008041381836
2023-09-28 23:26:39,603 - utils - INFO -  epoch: 436, all client loss: [0.5680555701255798, 0.48365724086761475], all pred client disparities: [0.011044442653656006, 0.0025439858436584473], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7789909243583679],  alpha_performance: tensor([0.5803, 0.4197], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:39,824 - utils - INFO - stage1_gradient_single_runtime: 0.0022325515747070312
2023-09-28 23:26:39,825 - utils - INFO -  epoch: 437, all client loss: [0.5682236552238464, 0.48349034786224365], all pred client disparities: [0.010726392269134521, 0.0024162381887435913], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7790220379829407],  alpha_performance: tensor([0.5825, 0.4175], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:40,053 - utils - INFO - stage1_gradient_single_runtime: 0.0022606849670410156
2023-09-28 23:26:40,054 - utils - INFO -  epoch: 438, all client loss: [0.5683895945549011, 0.4833255112171173], all pred client disparities: [0.01040235161781311, 0.0022933930158615112], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7790220379829407],  alpha_performance: tensor([0.5846, 0.4154], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:40,279 - utils - INFO - stage1_gradient_single_runtime: 0.002258777618408203
2023-09-28 23:26:40,280 - utils - INFO -  epoch: 439, all client loss: [0.5685535669326782, 0.48316270112991333], all pred client disparities: [0.010072052478790283, 0.0021753311157226562], all client disparities: [0.021014481782913208, 0.01160278171300888], all client accs: [0.7481840252876282, 0.7762846946716309],  alpha_performance: tensor([0.5868, 0.4132], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:40,362 - utils - INFO - valid: True, epoch: 439, loss: [0.5967323780059814, 0.4819906949996948], accuracy: [0.6961326003074646, 0.7789440751075745], mean_accuracy:0.7375383377075195,variance_accuracy:0.04140573740005493, disparity: [0.04545453190803528, 0.013055920600891113], mean_disparity:0.029255226254463196,variance_disparity:0.016199305653572083, pred_disparity: [0.036478400230407715, 0.003651335835456848]
2023-09-28 23:26:40,491 - utils - INFO - global_valid: True, epoch: 439,  global_loss: 0.48326635360717773, global_accuracy: 0.8093436572203541,  global_disparity:0.014088407158851624, global_pred_disparity: 0.005222693085670471,
2023-09-28 23:26:40,722 - utils - INFO - stage1_gradient_single_runtime: 0.002259969711303711
2023-09-28 23:26:40,723 - utils - INFO -  epoch: 440, all client loss: [0.5687155723571777, 0.4830018877983093], all pred client disparities: [0.009735554456710815, 0.0020619481801986694], all client disparities: [0.021014481782913208, 0.011675871908664703], all client accs: [0.7481840252876282, 0.7762535810470581],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:40,952 - utils - INFO - stage1_gradient_single_runtime: 0.002270936965942383
2023-09-28 23:26:40,953 - utils - INFO -  epoch: 441, all client loss: [0.567674994468689, 0.4829673171043396], all pred client disparities: [0.01091468334197998, 0.0024037957191467285], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7457627654075623, 0.7790220379829407],  alpha_performance: tensor([0.5812, 0.4188], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:41,189 - utils - INFO - stage1_gradient_single_runtime: 0.0026454925537109375
2023-09-28 23:26:41,189 - utils - INFO -  epoch: 442, all client loss: [0.567839503288269, 0.4828038811683655], all pred client disparities: [0.010600060224533081, 0.00228002667427063], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7457627654075623, 0.7790220379829407],  alpha_performance: tensor([0.5834, 0.4166], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:41,429 - utils - INFO - stage1_gradient_single_runtime: 0.0024852752685546875
2023-09-28 23:26:41,430 - utils - INFO -  epoch: 443, all client loss: [0.5680020451545715, 0.4826425313949585], all pred client disparities: [0.010279357433319092, 0.002160981297492981], all client disparities: [0.021014481782913208, 0.011456608772277832], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5855, 0.4145], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:41,679 - utils - INFO - stage1_gradient_single_runtime: 0.002131223678588867
2023-09-28 23:26:41,679 - utils - INFO -  epoch: 444, all client loss: [0.5681625604629517, 0.4824830889701843], all pred client disparities: [0.009952723979949951, 0.0020465999841690063], all client disparities: [0.021014481782913208, 0.011675871908664703], all client accs: [0.7481840252876282, 0.7762535810470581],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:41,910 - utils - INFO - stage1_gradient_single_runtime: 0.002249479293823242
2023-09-28 23:26:41,911 - utils - INFO -  epoch: 445, all client loss: [0.5671337246894836, 0.48244836926460266], all pred client disparities: [0.011107832193374634, 0.002387985587120056], all client disparities: [0.019202888011932373, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7789909243583679],  alpha_performance: tensor([0.5799, 0.4201], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:42,139 - utils - INFO - stage1_gradient_single_runtime: 0.0022437572479248047
2023-09-28 23:26:42,140 - utils - INFO -  epoch: 446, all client loss: [0.5672968626022339, 0.48228639364242554], all pred client disparities: [0.010802507400512695, 0.0022632479667663574], all client disparities: [0.021014481782913208, 0.005139701068401337], all client accs: [0.7457627654075623, 0.7790220379829407],  alpha_performance: tensor([0.5820, 0.4180], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:42,370 - utils - INFO - stage1_gradient_single_runtime: 0.0022623538970947266
2023-09-28 23:26:42,370 - utils - INFO -  epoch: 447, all client loss: [0.567457914352417, 0.48212647438049316], all pred client disparities: [0.010491371154785156, 0.002143263816833496], all client disparities: [0.021014481782913208, 0.011456608772277832], all client accs: [0.7457627654075623, 0.7763468623161316],  alpha_performance: tensor([0.5841, 0.4159], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:42,596 - utils - INFO - stage1_gradient_single_runtime: 0.0022323131561279297
2023-09-28 23:26:42,597 - utils - INFO -  epoch: 448, all client loss: [0.5676169395446777, 0.4819685220718384], all pred client disparities: [0.010174453258514404, 0.0020279139280319214], all client disparities: [0.021014481782913208, 0.011529698967933655], all client accs: [0.7457627654075623, 0.7763158082962036],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:42,829 - utils - INFO - stage1_gradient_single_runtime: 0.0020635128021240234
2023-09-28 23:26:42,830 - utils - INFO -  epoch: 449, all client loss: [0.5677741169929504, 0.4818124771118164], all pred client disparities: [0.009851545095443726, 0.0019170641899108887], all client disparities: [0.021014481782913208, 0.011968225240707397], all client accs: [0.7457627654075623, 0.7767512798309326],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:43,058 - utils - INFO - stage1_gradient_single_runtime: 0.0026869773864746094
2023-09-28 23:26:43,060 - utils - INFO -  epoch: 450, all client loss: [0.5667494535446167, 0.4817788302898407], all pred client disparities: [0.01100650429725647, 0.0022554099559783936], all client disparities: [0.019202888011932373, 0.005139701068401337], all client accs: [0.7481840252876282, 0.7789909243583679],  alpha_performance: tensor([0.5807, 0.4193], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:43,294 - utils - INFO - stage1_gradient_single_runtime: 0.0031239986419677734
2023-09-28 23:26:43,295 - utils - INFO -  epoch: 451, all client loss: [0.5669091939926147, 0.481620192527771], all pred client disparities: [0.010704874992370605, 0.002134263515472412], all client disparities: [0.019202888011932373, 0.011456608772277832], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5827, 0.4173], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:43,529 - utils - INFO - stage1_gradient_single_runtime: 0.0027086734771728516
2023-09-28 23:26:43,531 - utils - INFO -  epoch: 452, all client loss: [0.5670669674873352, 0.4814635217189789], all pred client disparities: [0.010397464036941528, 0.0020177066326141357], all client disparities: [0.021014481782913208, 0.011529698967933655], all client accs: [0.7457627654075623, 0.7763158082962036],  alpha_performance: tensor([0.5848, 0.4152], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:43,765 - utils - INFO - stage1_gradient_single_runtime: 0.0026781558990478516
2023-09-28 23:26:43,766 - utils - INFO -  epoch: 453, all client loss: [0.5672227740287781, 0.48130878806114197], all pred client disparities: [0.010084390640258789, 0.0019056200981140137], all client disparities: [0.021014481782913208, 0.01160278171300888], all client accs: [0.7457627654075623, 0.7762846946716309],  alpha_performance: tensor([0.5868, 0.4132], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:43,992 - utils - INFO - stage1_gradient_single_runtime: 0.002081632614135742
2023-09-28 23:26:43,992 - utils - INFO -  epoch: 454, all client loss: [0.5673767328262329, 0.4811559021472931], all pred client disparities: [0.00976550579071045, 0.0017979443073272705], all client disparities: [0.021014481782913208, 0.01204131543636322], all client accs: [0.7457627654075623, 0.7767201662063599],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:44,212 - utils - INFO - stage1_gradient_single_runtime: 0.002246379852294922
2023-09-28 23:26:44,213 - utils - INFO -  epoch: 455, all client loss: [0.5663567781448364, 0.4811231791973114], all pred client disparities: [0.010919123888015747, 0.0021335333585739136], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5813, 0.4187], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:44,434 - utils - INFO - stage1_gradient_single_runtime: 0.002235889434814453
2023-09-28 23:26:44,435 - utils - INFO -  epoch: 456, all client loss: [0.5665133595466614, 0.4809676706790924], all pred client disparities: [0.010621249675750732, 0.002015545964241028], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5833, 0.4167], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:44,658 - utils - INFO - stage1_gradient_single_runtime: 0.0022623538970947266
2023-09-28 23:26:44,659 - utils - INFO -  epoch: 457, all client loss: [0.5666680932044983, 0.4808140695095062], all pred client disparities: [0.010317832231521606, 0.0019020885229110718], all client disparities: [0.019202888011932373, 0.01160278171300888], all client accs: [0.7481840252876282, 0.7762846946716309],  alpha_performance: tensor([0.5853, 0.4147], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:44,885 - utils - INFO - stage1_gradient_single_runtime: 0.0022842884063720703
2023-09-28 23:26:44,886 - utils - INFO -  epoch: 458, all client loss: [0.5668208599090576, 0.48066237568855286], all pred client disparities: [0.010008811950683594, 0.0017929524183273315], all client disparities: [0.021014481782913208, 0.011895142495632172], all client accs: [0.7457627654075623, 0.7767823934555054],  alpha_performance: tensor([0.5874, 0.4126], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:45,109 - utils - INFO - stage1_gradient_single_runtime: 0.0020728111267089844
2023-09-28 23:26:45,110 - utils - INFO -  epoch: 459, all client loss: [0.5669718384742737, 0.48051247000694275], all pred client disparities: [0.009694039821624756, 0.0016880929470062256], all client disparities: [0.021014481782913208, 0.010255798697471619], all client accs: [0.7457627654075623, 0.7764402031898499],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:45,193 - utils - INFO - valid: True, epoch: 459, loss: [0.594878613948822, 0.47946178913116455], accuracy: [0.6961326003074646, 0.7790062427520752], mean_accuracy:0.7375694215297699,variance_accuracy:0.0414368212223053, disparity: [0.04999998211860657, 0.012464210391044617], mean_disparity:0.031232096254825592,variance_disparity:0.018767885863780975, pred_disparity: [0.03508734703063965, 0.0034888535737991333]
2023-09-28 23:26:45,324 - utils - INFO - global_valid: True, epoch: 459,  global_loss: 0.4807449281215668, global_accuracy: 0.8100591406044662,  global_disparity:0.013372063636779785, global_pred_disparity: 0.0051272958517074585,
2023-09-28 23:26:45,549 - utils - INFO - stage1_gradient_single_runtime: 0.0022504329681396484
2023-09-28 23:26:45,550 - utils - INFO -  epoch: 460, all client loss: [0.5659569501876831, 0.4804805815219879], all pred client disparities: [0.010845303535461426, 0.002021089196205139], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5819, 0.4181], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:45,775 - utils - INFO - stage1_gradient_single_runtime: 0.0022573471069335938
2023-09-28 23:26:45,775 - utils - INFO -  epoch: 461, all client loss: [0.5661105513572693, 0.4803280234336853], all pred client disparities: [0.010551363229751587, 0.001905977725982666], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5839, 0.4161], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:46,005 - utils - INFO - stage1_gradient_single_runtime: 0.0022575855255126953
2023-09-28 23:26:46,005 - utils - INFO -  epoch: 462, all client loss: [0.5662623047828674, 0.4801773130893707], all pred client disparities: [0.0102519690990448, 0.001795247197151184], all client disparities: [0.019202888011932373, 0.011822052299976349], all client accs: [0.7481840252876282, 0.7768135070800781],  alpha_performance: tensor([0.5859, 0.4141], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:46,229 - utils - INFO - stage1_gradient_single_runtime: 0.002079486846923828
2023-09-28 23:26:46,229 - utils - INFO -  epoch: 463, all client loss: [0.5664122700691223, 0.4800284206867218], all pred client disparities: [0.009947150945663452, 0.0016887784004211426], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:46,455 - utils - INFO - stage1_gradient_single_runtime: 0.0022640228271484375
2023-09-28 23:26:46,456 - utils - INFO -  epoch: 464, all client loss: [0.5654098391532898, 0.4799962043762207], all pred client disparities: [0.011071979999542236, 0.0020218491554260254], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7762535810470581],  alpha_performance: tensor([0.5804, 0.4196], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:46,680 - utils - INFO - stage1_gradient_single_runtime: 0.0022699832916259766
2023-09-28 23:26:46,681 - utils - INFO -  epoch: 465, all client loss: [0.56556236743927, 0.47984471917152405], all pred client disparities: [0.010787397623062134, 0.001905202865600586], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7762846946716309],  alpha_performance: tensor([0.5823, 0.4177], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:46,906 - utils - INFO - stage1_gradient_single_runtime: 0.002283334732055664
2023-09-28 23:26:46,907 - utils - INFO -  epoch: 466, all client loss: [0.565713107585907, 0.4796951115131378], all pred client disparities: [0.010497599840164185, 0.0017929226160049438], all client disparities: [0.019202888011932373, 0.01160278171300888], all client accs: [0.7481840252876282, 0.7762535810470581],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:47,132 - utils - INFO - stage1_gradient_single_runtime: 0.002249479293823242
2023-09-28 23:26:47,133 - utils - INFO -  epoch: 467, all client loss: [0.5658618807792664, 0.4795473515987396], all pred client disparities: [0.010202527046203613, 0.0016849040985107422], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:47,358 - utils - INFO - stage1_gradient_single_runtime: 0.002065420150756836
2023-09-28 23:26:47,359 - utils - INFO -  epoch: 468, all client loss: [0.5660088658332825, 0.4794013500213623], all pred client disparities: [0.009902030229568481, 0.0015810281038284302], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:47,584 - utils - INFO - stage1_gradient_single_runtime: 0.0022606849670410156
2023-09-28 23:26:47,585 - utils - INFO -  epoch: 469, all client loss: [0.5650116801261902, 0.4793698787689209], all pred client disparities: [0.011023014783859253, 0.0019116848707199097], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7762224674224854],  alpha_performance: tensor([0.5808, 0.4192], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:47,814 - utils - INFO - stage1_gradient_single_runtime: 0.002260923385620117
2023-09-28 23:26:47,815 - utils - INFO -  epoch: 470, all client loss: [0.5651613473892212, 0.4792212247848511], all pred client disparities: [0.010742515325546265, 0.0017976462841033936], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7762846946716309],  alpha_performance: tensor([0.5827, 0.4173], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:48,000 - utils - INFO - stage1_gradient_single_runtime: 0.002259969711303711
2023-09-28 23:26:48,001 - utils - INFO -  epoch: 471, all client loss: [0.5653092265129089, 0.4790744483470917], all pred client disparities: [0.010456979274749756, 0.0016879141330718994], all client disparities: [0.019202888011932373, 0.011895142495632172], all client accs: [0.7481840252876282, 0.7767512798309326],  alpha_performance: tensor([0.5846, 0.4154], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:48,231 - utils - INFO - stage1_gradient_single_runtime: 0.0022635459899902344
2023-09-28 23:26:48,232 - utils - INFO -  epoch: 472, all client loss: [0.5654551982879639, 0.47892946004867554], all pred client disparities: [0.010166257619857788, 0.001582309603691101], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5865, 0.4135], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:48,465 - utils - INFO - stage1_gradient_single_runtime: 0.002177000045776367
2023-09-28 23:26:48,466 - utils - INFO -  epoch: 473, all client loss: [0.5655995011329651, 0.4787862002849579], all pred client disparities: [0.009870260953903198, 0.0014807432889938354], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:48,696 - utils - INFO - stage1_gradient_single_runtime: 0.0022509098052978516
2023-09-28 23:26:48,697 - utils - INFO -  epoch: 474, all client loss: [0.5646079778671265, 0.4787553548812866], all pred client disparities: [0.010986328125, 0.0018091201782226562], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7762224674224854],  alpha_performance: tensor([0.5811, 0.4189], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:48,926 - utils - INFO - stage1_gradient_single_runtime: 0.002272367477416992
2023-09-28 23:26:48,927 - utils - INFO -  epoch: 475, all client loss: [0.5647549033164978, 0.4786094129085541], all pred client disparities: [0.010710150003433228, 0.0016974806785583496], all client disparities: [0.019202888011932373, 0.011822052299976349], all client accs: [0.7481840252876282, 0.7767512798309326],  alpha_performance: tensor([0.5830, 0.4170], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:49,171 - utils - INFO - stage1_gradient_single_runtime: 0.0023102760314941406
2023-09-28 23:26:49,172 - utils - INFO -  epoch: 476, all client loss: [0.5649001002311707, 0.47846531867980957], all pred client disparities: [0.010428935289382935, 0.0015899837017059326], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5849, 0.4151], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:49,405 - utils - INFO - stage1_gradient_single_runtime: 0.0022711753845214844
2023-09-28 23:26:49,406 - utils - INFO -  epoch: 477, all client loss: [0.5650434494018555, 0.47832298278808594], all pred client disparities: [0.010142743587493896, 0.0014865398406982422], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5868, 0.4132], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:49,639 - utils - INFO - stage1_gradient_single_runtime: 0.002094745635986328
2023-09-28 23:26:49,639 - utils - INFO -  epoch: 478, all client loss: [0.5651851296424866, 0.478182315826416], all pred client disparities: [0.0098513662815094, 0.0013870447874069214], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:49,871 - utils - INFO - stage1_gradient_single_runtime: 0.0022470951080322266
2023-09-28 23:26:49,872 - utils - INFO -  epoch: 479, all client loss: [0.5641995668411255, 0.47815200686454773], all pred client disparities: [0.0109616219997406, 0.0017133355140686035], all client disparities: [0.019202888011932373, 0.011529698967933655], all client accs: [0.7481840252876282, 0.7762224674224854],  alpha_performance: tensor([0.5814, 0.4186], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:49,956 - utils - INFO - valid: True, epoch: 479, loss: [0.5938212275505066, 0.47697514295578003], accuracy: [0.6961326003074646, 0.7795031070709229], mean_accuracy:0.7378178536891937,variance_accuracy:0.041685253381729126, disparity: [0.04999998211860657, 0.014522194862365723], mean_disparity:0.032261088490486145,variance_disparity:0.017738893628120422, pred_disparity: [0.0350324809551239, 0.003765478730201721]
2023-09-28 23:26:50,090 - utils - INFO - global_valid: True, epoch: 479,  global_loss: 0.4782741665840149, global_accuracy: 0.8103577291463191,  global_disparity:0.015349626541137695, global_pred_disparity: 0.005410492420196533,
2023-09-28 23:26:50,317 - utils - INFO - stage1_gradient_single_runtime: 0.0022859573364257812
2023-09-28 23:26:50,318 - utils - INFO -  epoch: 480, all client loss: [0.5643438696861267, 0.4780086278915405], all pred client disparities: [0.010689795017242432, 0.0016038119792938232], all client disparities: [0.019202888011932373, 0.011895142495632172], all client accs: [0.7481840252876282, 0.7767201662063599],  alpha_performance: tensor([0.5832, 0.4168], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:50,547 - utils - INFO - stage1_gradient_single_runtime: 0.0022513866424560547
2023-09-28 23:26:50,548 - utils - INFO -  epoch: 481, all client loss: [0.5644865036010742, 0.477867066860199], all pred client disparities: [0.010413140058517456, 0.0014983266592025757], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5851, 0.4149], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:50,779 - utils - INFO - stage1_gradient_single_runtime: 0.002268552780151367
2023-09-28 23:26:50,780 - utils - INFO -  epoch: 482, all client loss: [0.5646274089813232, 0.4777272343635559], all pred client disparities: [0.010131478309631348, 0.0013968199491500854], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5869, 0.4131], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:51,016 - utils - INFO - stage1_gradient_single_runtime: 0.0020906925201416016
2023-09-28 23:26:51,017 - utils - INFO -  epoch: 483, all client loss: [0.5647665858268738, 0.47758907079696655], all pred client disparities: [0.009844869375228882, 0.0012991726398468018], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:51,240 - utils - INFO - stage1_gradient_single_runtime: 0.0022516250610351562
2023-09-28 23:26:51,241 - utils - INFO -  epoch: 484, all client loss: [0.5637872219085693, 0.47755923867225647], all pred client disparities: [0.01094844937324524, 0.0016235411167144775], all client disparities: [0.019202888011932373, 0.011822052299976349], all client accs: [0.7481840252876282, 0.7766890525817871],  alpha_performance: tensor([0.5816, 0.4184], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:51,478 - utils - INFO - stage1_gradient_single_runtime: 0.0022842884063720703
2023-09-28 23:26:51,479 - utils - INFO -  epoch: 485, all client loss: [0.5639291405677795, 0.47741833329200745], all pred client disparities: [0.010681092739105225, 0.001515895128250122], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5834, 0.4166], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:51,705 - utils - INFO - stage1_gradient_single_runtime: 0.002287149429321289
2023-09-28 23:26:51,706 - utils - INFO -  epoch: 486, all client loss: [0.5640692710876465, 0.4772792160511017], all pred client disparities: [0.010408997535705566, 0.0014121979475021362], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5852, 0.4148], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:51,934 - utils - INFO - stage1_gradient_single_runtime: 0.002294301986694336
2023-09-28 23:26:51,934 - utils - INFO -  epoch: 487, all client loss: [0.5642077326774597, 0.47714173793792725], all pred client disparities: [0.01013210415840149, 0.0013124197721481323], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5870, 0.4130], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:52,160 - utils - INFO - stage1_gradient_single_runtime: 0.002071380615234375
2023-09-28 23:26:52,161 - utils - INFO -  epoch: 488, all client loss: [0.564344584941864, 0.47700586915016174], all pred client disparities: [0.00985032320022583, 0.001216396689414978], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:52,388 - utils - INFO - stage1_gradient_single_runtime: 0.0022585391998291016
2023-09-28 23:26:52,389 - utils - INFO -  epoch: 489, all client loss: [0.5633717179298401, 0.47697651386260986], all pred client disparities: [0.010946333408355713, 0.001538991928100586], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5817, 0.4183], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:52,615 - utils - INFO - stage1_gradient_single_runtime: 0.0022537708282470703
2023-09-28 23:26:52,615 - utils - INFO -  epoch: 490, all client loss: [0.563511312007904, 0.47683796286582947], all pred client disparities: [0.010683625936508179, 0.0014330297708511353], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5835, 0.4165], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:52,839 - utils - INFO - stage1_gradient_single_runtime: 0.0022487640380859375
2023-09-28 23:26:52,840 - utils - INFO -  epoch: 491, all client loss: [0.5636491179466248, 0.47670114040374756], all pred client disparities: [0.010416269302368164, 0.0013309568166732788], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5852, 0.4148], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:53,066 - utils - INFO - stage1_gradient_single_runtime: 0.0022428035736083984
2023-09-28 23:26:53,067 - utils - INFO -  epoch: 492, all client loss: [0.5637853145599365, 0.476565957069397], all pred client disparities: [0.010144174098968506, 0.0012326836585998535], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5870, 0.4130], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:53,296 - utils - INFO - stage1_gradient_single_runtime: 0.0020580291748046875
2023-09-28 23:26:53,296 - utils - INFO -  epoch: 493, all client loss: [0.5639198422431946, 0.47643235325813293], all pred client disparities: [0.009867280721664429, 0.0011381059885025024], all client disparities: [0.019202888011932373, 0.010182715952396393], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:53,521 - utils - INFO - stage1_gradient_single_runtime: 0.0022869110107421875
2023-09-28 23:26:53,522 - utils - INFO -  epoch: 494, all client loss: [0.5629537105560303, 0.4764033555984497], all pred client disparities: [0.010954976081848145, 0.001459077000617981], all client disparities: [0.019202888011932373, 0.010036535561084747], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5817, 0.4183], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:53,751 - utils - INFO - stage1_gradient_single_runtime: 0.002244710922241211
2023-09-28 23:26:53,752 - utils - INFO -  epoch: 495, all client loss: [0.5630910396575928, 0.4762670397758484], all pred client disparities: [0.010696947574615479, 0.0013545900583267212], all client disparities: [0.019202888011932373, 0.010036535561084747], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5835, 0.4165], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:53,984 - utils - INFO - stage1_gradient_single_runtime: 0.002234220504760742
2023-09-28 23:26:53,985 - utils - INFO -  epoch: 496, all client loss: [0.563226580619812, 0.47613242268562317], all pred client disparities: [0.010434329509735107, 0.0012539476156234741], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5852, 0.4148], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:54,213 - utils - INFO - stage1_gradient_single_runtime: 0.002254009246826172
2023-09-28 23:26:54,214 - utils - INFO -  epoch: 497, all client loss: [0.5633606314659119, 0.4759994149208069], all pred client disparities: [0.010167092084884644, 0.001157030463218689], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5870, 0.4130], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:54,439 - utils - INFO - stage1_gradient_single_runtime: 0.0020608901977539062
2023-09-28 23:26:54,439 - utils - INFO -  epoch: 498, all client loss: [0.563493013381958, 0.47586801648139954], all pred client disparities: [0.009895265102386475, 0.0010637342929840088], all client disparities: [0.019202888011932373, 0.010255798697471619], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:54,666 - utils - INFO - stage1_gradient_single_runtime: 0.0022735595703125
2023-09-28 23:26:54,667 - utils - INFO -  epoch: 499, all client loss: [0.5625337362289429, 0.4758392870426178], all pred client disparities: [0.010973900556564331, 0.0013832151889801025], all client disparities: [0.019202888011932373, 0.010036535561084747], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5817, 0.4183], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:54,751 - utils - INFO - valid: True, epoch: 499, loss: [0.5927252173423767, 0.4746589958667755], accuracy: [0.6961326003074646, 0.7789440751075745], mean_accuracy:0.7375383377075195,variance_accuracy:0.04140573740005493, disparity: [0.04999998211860657, 0.01542278379201889], mean_disparity:0.03271138295531273,variance_disparity:0.01728859916329384, pred_disparity: [0.034738630056381226, 0.0039405375719070435]
2023-09-28 23:26:54,878 - utils - INFO - global_valid: True, epoch: 499,  global_loss: 0.4759715795516968, global_accuracy: 0.8107634373730398,  global_disparity:0.016237400472164154, global_pred_disparity: 0.005602449178695679,
2023-09-28 23:26:55,106 - utils - INFO - stage1_gradient_single_runtime: 0.002307415008544922
2023-09-28 23:26:55,107 - utils - INFO -  epoch: 500, all client loss: [0.5626688599586487, 0.4757051467895508], all pred client disparities: [0.010720580816268921, 0.001280069351196289], all client disparities: [0.019202888011932373, 0.010036535561084747], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5834, 0.4166], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:55,352 - utils - INFO - stage1_gradient_single_runtime: 0.0026547908782958984
2023-09-28 23:26:55,353 - utils - INFO -  epoch: 501, all client loss: [0.5628023743629456, 0.47557270526885986], all pred client disparities: [0.010462820529937744, 0.0011806637048721313], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([0.5851, 0.4149], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:55,581 - utils - INFO - stage1_gradient_single_runtime: 0.0025081634521484375
2023-09-28 23:26:55,582 - utils - INFO -  epoch: 502, all client loss: [0.5629342198371887, 0.47544175386428833], all pred client disparities: [0.010200560092926025, 0.0010849088430404663], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([0.5869, 0.4131], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:55,846 - utils - INFO - stage1_gradient_single_runtime: 0.002357006072998047
2023-09-28 23:26:55,847 - utils - INFO -  epoch: 503, all client loss: [0.5630645155906677, 0.4753124713897705], all pred client disparities: [0.009933799505233765, 0.0009927451610565186], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:56,080 - utils - INFO - stage1_gradient_single_runtime: 0.00257110595703125
2023-09-28 23:26:56,081 - utils - INFO -  epoch: 504, all client loss: [0.5621123909950256, 0.47528401017189026], all pred client disparities: [0.01100274920463562, 0.0013108551502227783], all client disparities: [0.019202888011932373, 0.010036535561084747], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5816, 0.4184], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:56,315 - utils - INFO - stage1_gradient_single_runtime: 0.002567291259765625
2023-09-28 23:26:56,316 - utils - INFO -  epoch: 505, all client loss: [0.5622454881668091, 0.4751519560813904], all pred client disparities: [0.010754168033599854, 0.001208871603012085], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5833, 0.4167], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:56,549 - utils - INFO - stage1_gradient_single_runtime: 0.002566099166870117
2023-09-28 23:26:56,550 - utils - INFO -  epoch: 506, all client loss: [0.5623767971992493, 0.4750215411186218], all pred client disparities: [0.010501354932785034, 0.001110568642616272], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5850, 0.4150], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:56,783 - utils - INFO - stage1_gradient_single_runtime: 0.002242565155029297
2023-09-28 23:26:56,785 - utils - INFO -  epoch: 507, all client loss: [0.5625066161155701, 0.47489267587661743], all pred client disparities: [0.010244101285934448, 0.0010158717632293701], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5867, 0.4133], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:57,010 - utils - INFO - stage1_gradient_single_runtime: 0.0020911693572998047
2023-09-28 23:26:57,011 - utils - INFO -  epoch: 508, all client loss: [0.5626348853111267, 0.4747653901576996], all pred client disparities: [0.009982496500015259, 0.0009247064590454102], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:57,234 - utils - INFO - stage1_gradient_single_runtime: 0.002249479293823242
2023-09-28 23:26:57,235 - utils - INFO -  epoch: 509, all client loss: [0.5616899728775024, 0.47473713755607605], all pred client disparities: [0.011041045188903809, 0.0012415200471878052], all client disparities: [0.019202888011932373, 0.010036535561084747], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5815, 0.4185], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:57,458 - utils - INFO - stage1_gradient_single_runtime: 0.002247333526611328
2023-09-28 23:26:57,459 - utils - INFO -  epoch: 510, all client loss: [0.5618210434913635, 0.47460705041885376], all pred client disparities: [0.010797351598739624, 0.0011405646800994873], all client disparities: [0.019202888011932373, 0.01010962575674057], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5831, 0.4169], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:57,679 - utils - INFO - stage1_gradient_single_runtime: 0.002229928970336914
2023-09-28 23:26:57,680 - utils - INFO -  epoch: 511, all client loss: [0.5619503855705261, 0.47447866201400757], all pred client disparities: [0.010549426078796387, 0.0010432451963424683], all client disparities: [0.019202888011932373, 0.009389162063598633], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5848, 0.4152], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:57,900 - utils - INFO - stage1_gradient_single_runtime: 0.0022423267364501953
2023-09-28 23:26:57,901 - utils - INFO -  epoch: 512, all client loss: [0.5620782375335693, 0.47435176372528076], all pred client disparities: [0.010297298431396484, 0.0009494423866271973], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5864, 0.4136], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:58,122 - utils - INFO - stage1_gradient_single_runtime: 0.0022673606872558594
2023-09-28 23:26:58,122 - utils - INFO -  epoch: 513, all client loss: [0.5622045397758484, 0.4742264449596405], all pred client disparities: [0.010040909051895142, 0.0008591115474700928], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5881, 0.4119], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:58,342 - utils - INFO - stage1_gradient_single_runtime: 0.002055644989013672
2023-09-28 23:26:58,343 - utils - INFO -  epoch: 514, all client loss: [0.5623293519020081, 0.47410252690315247], all pred client disparities: [0.009780079126358032, 0.0007721036672592163], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:58,567 - utils - INFO - stage1_gradient_single_runtime: 0.0022444725036621094
2023-09-28 23:26:58,568 - utils - INFO -  epoch: 515, all client loss: [0.5613860487937927, 0.47407519817352295], all pred client disparities: [0.010846734046936035, 0.0010854750871658325], all client disparities: [0.019202888011932373, 0.009389162063598633], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5830, 0.4170], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:58,791 - utils - INFO - stage1_gradient_single_runtime: 0.0022230148315429688
2023-09-28 23:26:58,791 - utils - INFO -  epoch: 516, all client loss: [0.5615137219429016, 0.47394850850105286], all pred client disparities: [0.01060381531715393, 0.0009887665510177612], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5846, 0.4154], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:59,015 - utils - INFO - stage1_gradient_single_runtime: 0.0022330284118652344
2023-09-28 23:26:59,016 - utils - INFO -  epoch: 517, all client loss: [0.561639666557312, 0.4738233983516693], all pred client disparities: [0.010356813669204712, 0.0008955895900726318], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:59,243 - utils - INFO - stage1_gradient_single_runtime: 0.002260923385620117
2023-09-28 23:26:59,244 - utils - INFO -  epoch: 518, all client loss: [0.5617642998695374, 0.47369974851608276], all pred client disparities: [0.010105609893798828, 0.0008057653903961182], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5878, 0.4122], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:59,468 - utils - INFO - stage1_gradient_single_runtime: 0.002070188522338867
2023-09-28 23:26:59,469 - utils - INFO -  epoch: 519, all client loss: [0.5618874430656433, 0.4735775589942932], all pred client disparities: [0.009850144386291504, 0.0007192641496658325], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:26:59,556 - utils - INFO - valid: True, epoch: 519, loss: [0.5916060209274292, 0.47249332070350647], accuracy: [0.6961326003074646, 0.7791925668716431], mean_accuracy:0.7376625835895538,variance_accuracy:0.04152998328208923, disparity: [0.04999998211860657, 0.015274859964847565], mean_disparity:0.032637421041727066,variance_disparity:0.0173625610768795, pred_disparity: [0.0342353880405426, 0.004031643271446228]
2023-09-28 23:26:59,681 - utils - INFO - global_valid: True, epoch: 519,  global_loss: 0.4738175868988037, global_accuracy: 0.8111005204180824,  global_disparity:0.016094133257865906, global_pred_disparity: 0.005719482898712158,
2023-09-28 23:26:59,914 - utils - INFO - stage1_gradient_single_runtime: 0.0022742748260498047
2023-09-28 23:26:59,915 - utils - INFO -  epoch: 520, all client loss: [0.5609520673751831, 0.47355031967163086], all pred client disparities: [0.010904669761657715, 0.0010317713022232056], all client disparities: [0.019202888011932373, 0.009389162063598633], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5827, 0.4173], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:00,143 - utils - INFO - stage1_gradient_single_runtime: 0.0022618770599365234
2023-09-28 23:27:00,144 - utils - INFO -  epoch: 521, all client loss: [0.5610779523849487, 0.47342532873153687], all pred client disparities: [0.010666817426681519, 0.0009356141090393066], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:00,375 - utils - INFO - stage1_gradient_single_runtime: 0.0022296905517578125
2023-09-28 23:27:00,375 - utils - INFO -  epoch: 522, all client loss: [0.561202347278595, 0.4733019173145294], all pred client disparities: [0.010424911975860596, 0.0008429139852523804], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5859, 0.4141], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:00,603 - utils - INFO - stage1_gradient_single_runtime: 0.0022454261779785156
2023-09-28 23:27:00,604 - utils - INFO -  epoch: 523, all client loss: [0.561325192451477, 0.4731799364089966], all pred client disparities: [0.010178953409194946, 0.000753551721572876], all client disparities: [0.019202888011932373, 0.009535335004329681], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5875, 0.4125], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:00,833 - utils - INFO - stage1_gradient_single_runtime: 0.0024597644805908203
2023-09-28 23:27:00,833 - utils - INFO -  epoch: 524, all client loss: [0.5614466071128845, 0.4730594754219055], all pred client disparities: [0.009928882122039795, 0.0006674677133560181], all client disparities: [0.019202888011932373, 0.009827695786952972], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:01,083 - utils - INFO - stage1_gradient_single_runtime: 0.0023698806762695312
2023-09-28 23:27:01,084 - utils - INFO -  epoch: 525, all client loss: [0.560519278049469, 0.47303223609924316], all pred client disparities: [0.010970830917358398, 0.0009791851043701172], all client disparities: [0.019202888011932373, 0.009242981672286987], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5824, 0.4176], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:01,323 - utils - INFO - stage1_gradient_single_runtime: 0.002256631851196289
2023-09-28 23:27:01,324 - utils - INFO -  epoch: 526, all client loss: [0.5606434941291809, 0.47290894389152527], all pred client disparities: [0.010738015174865723, 0.0008834898471832275], all client disparities: [0.019202888011932373, 0.009389162063598633], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5840, 0.4160], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:01,553 - utils - INFO - stage1_gradient_single_runtime: 0.0022819042205810547
2023-09-28 23:27:01,554 - utils - INFO -  epoch: 527, all client loss: [0.5607661604881287, 0.47278717160224915], all pred client disparities: [0.010501265525817871, 0.0007911771535873413], all client disparities: [0.019202888011932373, 0.009389162063598633], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5855, 0.4145], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:01,783 - utils - INFO - stage1_gradient_single_runtime: 0.0022513866424560547
2023-09-28 23:27:01,784 - utils - INFO -  epoch: 528, all client loss: [0.5608873963356018, 0.4726668894290924], all pred client disparities: [0.010260552167892456, 0.000702202320098877], all client disparities: [0.019202888011932373, 0.009827695786952972], all client accs: [0.7481840252876282, 0.7762846946716309],  alpha_performance: tensor([0.5871, 0.4129], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:02,010 - utils - INFO - stage1_gradient_single_runtime: 0.002265453338623047
2023-09-28 23:27:02,011 - utils - INFO -  epoch: 529, all client loss: [0.5610072016716003, 0.4725480377674103], all pred client disparities: [0.010015875101089478, 0.0006164312362670898], all client disparities: [0.019202888011932373, 0.009827695786952972], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5886, 0.4114], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:02,243 - utils - INFO - stage1_gradient_single_runtime: 0.0020995140075683594
2023-09-28 23:27:02,244 - utils - INFO -  epoch: 530, all client loss: [0.5611255764961243, 0.4724305272102356], all pred client disparities: [0.00976705551147461, 0.0005338042974472046], all client disparities: [0.019202888011932373, 0.009900778532028198], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:02,433 - utils - INFO - stage1_gradient_single_runtime: 0.002269268035888672
2023-09-28 23:27:02,434 - utils - INFO -  epoch: 531, all client loss: [0.5602006912231445, 0.47240400314331055], all pred client disparities: [0.010814070701599121, 0.000842556357383728], all client disparities: [0.019202888011932373, 0.00931607186794281], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5836, 0.4164], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:02,662 - utils - INFO - stage1_gradient_single_runtime: 0.0022819042205810547
2023-09-28 23:27:02,663 - utils - INFO -  epoch: 532, all client loss: [0.5603218674659729, 0.4722836911678314], all pred client disparities: [0.010582506656646729, 0.0007503777742385864], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5852, 0.4148], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:02,892 - utils - INFO - stage1_gradient_single_runtime: 0.002300262451171875
2023-09-28 23:27:02,893 - utils - INFO -  epoch: 533, all client loss: [0.5604416728019714, 0.4721648693084717], all pred client disparities: [0.010347038507461548, 0.0006614774465560913], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5867, 0.4133], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:03,121 - utils - INFO - stage1_gradient_single_runtime: 0.0022826194763183594
2023-09-28 23:27:03,122 - utils - INFO -  epoch: 534, all client loss: [0.5605599880218506, 0.4720474183559418], all pred client disparities: [0.010107725858688354, 0.0005757957696914673], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5882, 0.4118], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:03,349 - utils - INFO - stage1_gradient_single_runtime: 0.0020928382873535156
2023-09-28 23:27:03,350 - utils - INFO -  epoch: 535, all client loss: [0.5606769323348999, 0.4719313979148865], all pred client disparities: [0.009864449501037598, 0.0004932433366775513], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:03,575 - utils - INFO - stage1_gradient_single_runtime: 0.0022492408752441406
2023-09-28 23:27:03,575 - utils - INFO -  epoch: 536, all client loss: [0.5597606301307678, 0.47190478444099426], all pred client disparities: [0.01089748740196228, 0.0008015185594558716], all client disparities: [0.019202888011932373, 0.010036535561084747], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5833, 0.4167], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:03,802 - utils - INFO - stage1_gradient_single_runtime: 0.002239704132080078
2023-09-28 23:27:03,803 - utils - INFO -  epoch: 537, all client loss: [0.5598803758621216, 0.4717859625816345], all pred client disparities: [0.010671049356460571, 0.0007094442844390869], all client disparities: [0.019202888011932373, 0.009608425199985504], all client accs: [0.7481840252876282, 0.7762846946716309],  alpha_performance: tensor([0.5847, 0.4153], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:04,028 - utils - INFO - stage1_gradient_single_runtime: 0.0022878646850585938
2023-09-28 23:27:04,029 - utils - INFO -  epoch: 538, all client loss: [0.5599986910820007, 0.4716686010360718], all pred client disparities: [0.010440915822982788, 0.0006206035614013672], all client disparities: [0.019202888011932373, 0.009608425199985504], all client accs: [0.7481840252876282, 0.7762846946716309],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:04,257 - utils - INFO - stage1_gradient_single_runtime: 0.002271413803100586
2023-09-28 23:27:04,258 - utils - INFO -  epoch: 539, all client loss: [0.5601155161857605, 0.47155264019966125], all pred client disparities: [0.010206937789916992, 0.0005349218845367432], all client disparities: [0.019202888011932373, 0.009608425199985504], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5878, 0.4122], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:04,348 - utils - INFO - valid: True, epoch: 539, loss: [0.5911531448364258, 0.4703627824783325], accuracy: [0.6961326003074646, 0.7796273231506348], mean_accuracy:0.7378799617290497,variance_accuracy:0.04174736142158508, disparity: [0.04999998211860657, 0.01542278379201889], mean_disparity:0.03271138295531273,variance_disparity:0.01728859916329384, pred_disparity: [0.03492307662963867, 0.004528790712356567]
2023-09-28 23:27:04,474 - utils - INFO - global_valid: True, epoch: 539,  global_loss: 0.47170567512512207, global_accuracy: 0.8112458418722756,  global_disparity:0.016237400472164154, global_pred_disparity: 0.006188884377479553,
2023-09-28 23:27:04,697 - utils - INFO - stage1_gradient_single_runtime: 0.0020761489868164062
2023-09-28 23:27:04,697 - utils - INFO -  epoch: 540, all client loss: [0.5602310299873352, 0.4714379906654358], all pred client disparities: [0.009969145059585571, 0.0004523247480392456], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:04,922 - utils - INFO - stage1_gradient_single_runtime: 0.0022673606872558594
2023-09-28 23:27:04,923 - utils - INFO -  epoch: 541, all client loss: [0.5593233108520508, 0.4714113473892212], all pred client disparities: [0.010987907648086548, 0.000760212482418865], all client disparities: [0.019202888011932373, 0.01162363588809967], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5828, 0.4172], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:05,147 - utils - INFO - stage1_gradient_single_runtime: 0.002260923385620117
2023-09-28 23:27:05,147 - utils - INFO -  epoch: 542, all client loss: [0.5594415664672852, 0.47129395604133606], all pred client disparities: [0.010766685009002686, 0.0006681680679321289], all client disparities: [0.019202888011932373, 0.010328888893127441], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5843, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:05,373 - utils - INFO - stage1_gradient_single_runtime: 0.002282857894897461
2023-09-28 23:27:05,374 - utils - INFO -  epoch: 543, all client loss: [0.5595583915710449, 0.4711780250072479], all pred client disparities: [0.010541766881942749, 0.0005793273448944092], all client disparities: [0.019202888011932373, 0.009608425199985504], all client accs: [0.7481840252876282, 0.7762846946716309],  alpha_performance: tensor([0.5857, 0.4143], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:05,596 - utils - INFO - stage1_gradient_single_runtime: 0.002287149429321289
2023-09-28 23:27:05,597 - utils - INFO -  epoch: 544, all client loss: [0.5596738457679749, 0.4710635244846344], all pred client disparities: [0.010313153266906738, 0.0004936158657073975], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5872, 0.4128], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:05,821 - utils - INFO - stage1_gradient_single_runtime: 0.002254962921142578
2023-09-28 23:27:05,821 - utils - INFO -  epoch: 545, all client loss: [0.5597878694534302, 0.47095033526420593], all pred client disparities: [0.010080844163894653, 0.00041094422340393066], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5887, 0.4113], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:06,041 - utils - INFO - stage1_gradient_single_runtime: 0.0020928382873535156
2023-09-28 23:27:06,042 - utils - INFO -  epoch: 546, all client loss: [0.5599006414413452, 0.47083842754364014], all pred client disparities: [0.009844779968261719, 0.0003312528133392334], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:06,267 - utils - INFO - stage1_gradient_single_runtime: 0.0022394657135009766
2023-09-28 23:27:06,267 - utils - INFO -  epoch: 547, all client loss: [0.5589961409568787, 0.47081223130226135], all pred client disparities: [0.010865986347198486, 0.0006365925073623657], all client disparities: [0.019202888011932373, 0.01162363588809967], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5838, 0.4162], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:06,490 - utils - INFO - stage1_gradient_single_runtime: 0.002274751663208008
2023-09-28 23:27:06,490 - utils - INFO -  epoch: 548, all client loss: [0.5591117143630981, 0.4706975817680359], all pred client disparities: [0.01064637303352356, 0.0005475133657455444], all client disparities: [0.019202888011932373, 0.010328888893127441], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5852, 0.4148], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:06,715 - utils - INFO - stage1_gradient_single_runtime: 0.002271413803100586
2023-09-28 23:27:06,716 - utils - INFO -  epoch: 549, all client loss: [0.559225857257843, 0.47058430314064026], all pred client disparities: [0.010423123836517334, 0.00046153366565704346], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5867, 0.4133], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:06,941 - utils - INFO - stage1_gradient_single_runtime: 0.002238035202026367
2023-09-28 23:27:06,942 - utils - INFO -  epoch: 550, all client loss: [0.5593386292457581, 0.4704723656177521], all pred client disparities: [0.01019623875617981, 0.0003785938024520874], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5881, 0.4119], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:07,167 - utils - INFO - stage1_gradient_single_runtime: 0.002054452896118164
2023-09-28 23:27:07,168 - utils - INFO -  epoch: 551, all client loss: [0.559450089931488, 0.47036176919937134], all pred client disparities: [0.009965747594833374, 0.0002985745668411255], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:07,393 - utils - INFO - stage1_gradient_single_runtime: 0.0022592544555664062
2023-09-28 23:27:07,394 - utils - INFO -  epoch: 552, all client loss: [0.5585545301437378, 0.4703353941440582], all pred client disparities: [0.010971516370773315, 0.0006038099527359009], all client disparities: [0.019202888011932373, 0.011477455496788025], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5833, 0.4167], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:07,616 - utils - INFO - stage1_gradient_single_runtime: 0.002231121063232422
2023-09-28 23:27:07,617 - utils - INFO -  epoch: 553, all client loss: [0.5586687922477722, 0.4702220857143402], all pred client disparities: [0.010757148265838623, 0.0005144625902175903], all client disparities: [0.019202888011932373, 0.01162363588809967], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5847, 0.4153], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:07,841 - utils - INFO - stage1_gradient_single_runtime: 0.0022559165954589844
2023-09-28 23:27:07,842 - utils - INFO -  epoch: 554, all client loss: [0.5587816834449768, 0.4701100289821625], all pred client disparities: [0.010539233684539795, 0.00042819976806640625], all client disparities: [0.019202888011932373, 0.010401979088783264], all client accs: [0.7481840252876282, 0.7763468623161316],  alpha_performance: tensor([0.5861, 0.4139], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:08,066 - utils - INFO - stage1_gradient_single_runtime: 0.0022187232971191406
2023-09-28 23:27:08,067 - utils - INFO -  epoch: 555, all client loss: [0.5588932037353516, 0.46999940276145935], all pred client disparities: [0.010317802429199219, 0.0003449469804763794], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763158082962036],  alpha_performance: tensor([0.5875, 0.4125], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:08,293 - utils - INFO - stage1_gradient_single_runtime: 0.0022385120391845703
2023-09-28 23:27:08,293 - utils - INFO -  epoch: 556, all client loss: [0.5590033531188965, 0.4698900580406189], all pred client disparities: [0.01009279489517212, 0.00026459991931915283], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5889, 0.4111], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:08,522 - utils - INFO - stage1_gradient_single_runtime: 0.0023343563079833984
2023-09-28 23:27:08,523 - utils - INFO -  epoch: 557, all client loss: [0.5591123700141907, 0.4697819650173187], all pred client disparities: [0.009864270687103271, 0.00018712878227233887], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:08,763 - utils - INFO - stage1_gradient_single_runtime: 0.002483844757080078
2023-09-28 23:27:08,764 - utils - INFO -  epoch: 558, all client loss: [0.5582205653190613, 0.46975597739219666], all pred client disparities: [0.010870814323425293, 0.0004901140928268433], all client disparities: [0.019202888011932373, 0.01162363588809967], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5841, 0.4159], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:09,001 - utils - INFO - stage1_gradient_single_runtime: 0.002233743667602539
2023-09-28 23:27:09,002 - utils - INFO -  epoch: 559, all client loss: [0.5583322644233704, 0.46964511275291443], all pred client disparities: [0.01065826416015625, 0.00040337443351745605], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5855, 0.4145], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:09,086 - utils - INFO - valid: True, epoch: 559, loss: [0.5899922847747803, 0.46845245361328125], accuracy: [0.7016574740409851, 0.7796273231506348], mean_accuracy:0.7406423985958099,variance_accuracy:0.03898492455482483, disparity: [0.016666650772094727, 0.015274859964847565], mean_disparity:0.015970755368471146,variance_disparity:0.0006958954036235809, pred_disparity: [0.034020811319351196, 0.0044884830713272095]
2023-09-28 23:27:09,211 - utils - INFO - global_valid: True, epoch: 559,  global_loss: 0.46980366110801697, global_accuracy: 0.8116442944810676,  global_disparity:0.017498627305030823, global_pred_disparity: 0.006191536784172058,
2023-09-28 23:27:09,435 - utils - INFO - stage1_gradient_single_runtime: 0.0022764205932617188
2023-09-28 23:27:09,436 - utils - INFO -  epoch: 560, all client loss: [0.5584426522254944, 0.46953555941581726], all pred client disparities: [0.010442256927490234, 0.00031957030296325684], all client disparities: [0.019202888011932373, 0.010401979088783264], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5869, 0.4131], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:09,664 - utils - INFO - stage1_gradient_single_runtime: 0.002267599105834961
2023-09-28 23:27:09,665 - utils - INFO -  epoch: 561, all client loss: [0.5585517287254333, 0.46942734718322754], all pred client disparities: [0.01022273302078247, 0.0002387315034866333], all client disparities: [0.019202888011932373, 0.009681515395641327], all client accs: [0.7481840252876282, 0.7763779759407043],  alpha_performance: tensor([0.5883, 0.4117], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:09,890 - utils - INFO - stage1_gradient_single_runtime: 0.002065420150756836
2023-09-28 23:27:09,891 - utils - INFO -  epoch: 562, all client loss: [0.5586596131324768, 0.4693203568458557], all pred client disparities: [0.00999981164932251, 0.00016070902347564697], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:10,121 - utils - INFO - stage1_gradient_single_runtime: 0.002299070358276367
2023-09-28 23:27:10,122 - utils - INFO -  epoch: 563, all client loss: [0.5577769875526428, 0.46929416060447693], all pred client disparities: [0.010989934206008911, 0.00046381354331970215], all client disparities: [0.019202888011932373, 0.011477455496788025], all client accs: [0.7481840252876282, 0.7764402031898499],  alpha_performance: tensor([0.5835, 0.4165], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:10,346 - utils - INFO - stage1_gradient_single_runtime: 0.002279043197631836
2023-09-28 23:27:10,346 - utils - INFO -  epoch: 564, all client loss: [0.5578875541687012, 0.4691844880580902], all pred client disparities: [0.01078265905380249, 0.0003765374422073364], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([0.5849, 0.4151], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:10,574 - utils - INFO - stage1_gradient_single_runtime: 0.0022482872009277344
2023-09-28 23:27:10,575 - utils - INFO -  epoch: 565, all client loss: [0.5579968094825745, 0.4690760672092438], all pred client disparities: [0.01057201623916626, 0.0002922564744949341], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:10,806 - utils - INFO - stage1_gradient_single_runtime: 0.0022623538970947266
2023-09-28 23:27:10,807 - utils - INFO -  epoch: 566, all client loss: [0.5581047534942627, 0.4689689576625824], all pred client disparities: [0.010358035564422607, 0.00021082162857055664], all client disparities: [0.019202888011932373, 0.010401979088783264], all client accs: [0.7481840252876282, 0.7764090895652771],  alpha_performance: tensor([0.5876, 0.4124], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:11,033 - utils - INFO - stage1_gradient_single_runtime: 0.002585887908935547
2023-09-28 23:27:11,034 - utils - INFO -  epoch: 567, all client loss: [0.5582114458084106, 0.4688631296157837], all pred client disparities: [0.01014062762260437, 0.00013227760791778564], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5889, 0.4111], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:11,254 - utils - INFO - stage1_gradient_single_runtime: 0.0021064281463623047
2023-09-28 23:27:11,255 - utils - INFO -  epoch: 568, all client loss: [0.5583168864250183, 0.4687584936618805], all pred client disparities: [0.009919852018356323, 5.647540092468262e-05], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:11,483 - utils - INFO - stage1_gradient_single_runtime: 0.0022306442260742188
2023-09-28 23:27:11,484 - utils - INFO -  epoch: 569, all client loss: [0.5574384331703186, 0.468732625246048], all pred client disparities: [0.010909229516983032, 0.00035756826400756836], all client disparities: [0.019202888011932373, 0.011477455496788025], all client accs: [0.7481840252876282, 0.7765646576881409],  alpha_performance: tensor([0.5842, 0.4158], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:11,704 - utils - INFO - stage1_gradient_single_runtime: 0.0022933483123779297
2023-09-28 23:27:11,705 - utils - INFO -  epoch: 570, all client loss: [0.5575466752052307, 0.4686252176761627], all pred client disparities: [0.0107039213180542, 0.00027257204055786133], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7765335440635681],  alpha_performance: tensor([0.5855, 0.4145], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:11,927 - utils - INFO - stage1_gradient_single_runtime: 0.0022754669189453125
2023-09-28 23:27:11,928 - utils - INFO -  epoch: 571, all client loss: [0.5576536059379578, 0.4685191214084625], all pred client disparities: [0.010495424270629883, 0.00019043684005737305], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7765335440635681],  alpha_performance: tensor([0.5869, 0.4131], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:12,150 - utils - INFO - stage1_gradient_single_runtime: 0.0022439956665039062
2023-09-28 23:27:12,151 - utils - INFO -  epoch: 572, all client loss: [0.5577592849731445, 0.468414306640625], all pred client disparities: [0.010283619165420532, 0.00011113286018371582], all client disparities: [0.019202888011932373, 0.010475069284439087], all client accs: [0.7481840252876282, 0.7765024304389954],  alpha_performance: tensor([0.5882, 0.4118], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:12,372 - utils - INFO - stage1_gradient_single_runtime: 0.0022678375244140625
2023-09-28 23:27:12,373 - utils - INFO -  epoch: 573, all client loss: [0.557863712310791, 0.468310683965683], all pred client disparities: [0.01006847620010376, 3.458559876889922e-05], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([0.5896, 0.4104], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:12,596 - utils - INFO - stage1_gradient_single_runtime: 0.002072572708129883
2023-09-28 23:27:12,597 - utils - INFO -  epoch: 574, all client loss: [0.557966947555542, 0.4682082533836365], all pred client disparities: [0.009849905967712402, 3.9249658584594727e-05], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7765646576881409],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:12,853 - utils - INFO - stage1_gradient_single_runtime: 0.003238201141357422
2023-09-28 23:27:12,854 - utils - INFO -  epoch: 575, all client loss: [0.55709308385849, 0.4681825637817383], all pred client disparities: [0.010837644338607788, 0.0002600550651550293], all client disparities: [0.019202888011932373, 0.011550545692443848], all client accs: [0.7481840252876282, 0.7765646576881409],  alpha_performance: tensor([0.5849, 0.4151], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:13,104 - utils - INFO - stage1_gradient_single_runtime: 0.002262592315673828
2023-09-28 23:27:13,105 - utils - INFO -  epoch: 576, all client loss: [0.5571991205215454, 0.4680773913860321], all pred client disparities: [0.01063448190689087, 0.00017705559730529785], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7765335440635681],  alpha_performance: tensor([0.5862, 0.4138], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:13,330 - utils - INFO - stage1_gradient_single_runtime: 0.0022852420806884766
2023-09-28 23:27:13,330 - utils - INFO -  epoch: 577, all client loss: [0.5573038458824158, 0.4679734408855438], all pred client disparities: [0.010428130626678467, 9.688735008239746e-05], all client disparities: [0.019202888011932373, 0.011769808828830719], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5875, 0.4125], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:13,553 - utils - INFO - stage1_gradient_single_runtime: 0.0022966861724853516
2023-09-28 23:27:13,554 - utils - INFO -  epoch: 578, all client loss: [0.5574073791503906, 0.46787071228027344], all pred client disparities: [0.010218501091003418, 1.9446020814939402e-05], all client disparities: [0.019202888011932373, 0.011769808828830719], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5888, 0.4112], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:13,778 - utils - INFO - stage1_gradient_single_runtime: 0.002300262451171875
2023-09-28 23:27:13,778 - utils - INFO -  epoch: 579, all client loss: [0.5575097799301147, 0.46776917576789856], all pred client disparities: [0.010005742311477661, 5.5328015150735155e-05], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7765646576881409],  alpha_performance: tensor([0.4415, 0.5585], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:13,862 - utils - INFO - valid: True, epoch: 579, loss: [0.589385986328125, 0.4667065441608429], accuracy: [0.7016574740409851, 0.7795652151107788], mean_accuracy:0.740611344575882,variance_accuracy:0.03895387053489685, disparity: [0.016666650772094727, 0.015718646347522736], mean_disparity:0.01619264855980873,variance_disparity:0.0004740022122859955, pred_disparity: [0.034672945737838745, 0.0045862942934036255]
2023-09-28 23:27:13,986 - utils - INFO - global_valid: True, epoch: 579,  global_loss: 0.4680704176425934, global_accuracy: 0.8118565683519983,  global_disparity:0.017928428947925568, global_pred_disparity: 0.006270289421081543,
2023-09-28 23:27:14,215 - utils - INFO - stage1_gradient_single_runtime: 0.002064228057861328
2023-09-28 23:27:14,216 - utils - INFO -  epoch: 580, all client loss: [0.5574849247932434, 0.4677946865558624], all pred client disparities: [0.009573042392730713, 0.00018070638179779053], all client disparities: [0.019202888011932373, 0.00975460559129715], all client accs: [0.7481840252876282, 0.7764713168144226],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:14,452 - utils - INFO - stage1_gradient_single_runtime: 0.0022509098052978516
2023-09-28 23:27:14,453 - utils - INFO -  epoch: 581, all client loss: [0.5566214919090271, 0.4677685797214508], all pred client disparities: [0.01056361198425293, 0.0004828125238418579], all client disparities: [0.019202888011932373, 0.011404365301132202], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5887, 0.4113], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:14,681 - utils - INFO - stage1_gradient_single_runtime: 0.0025916099548339844
2023-09-28 23:27:14,683 - utils - INFO -  epoch: 582, all client loss: [0.5567286014556885, 0.4676622748374939], all pred client disparities: [0.010359764099121094, 0.00039850175380706787], all client disparities: [0.019202888011932373, 0.011550545692443848], all client accs: [0.7481840252876282, 0.7765646576881409],  alpha_performance: tensor([0.5900, 0.4100], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:14,911 - utils - INFO - stage1_gradient_single_runtime: 0.002568960189819336
2023-09-28 23:27:14,913 - utils - INFO -  epoch: 583, all client loss: [0.5568344593048096, 0.4675573408603668], all pred client disparities: [0.010152816772460938, 0.00031700730323791504], all client disparities: [0.019202888011932373, 0.011769808828830719], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5913, 0.4087], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:15,153 - utils - INFO - stage1_gradient_single_runtime: 0.0020720958709716797
2023-09-28 23:27:15,153 - utils - INFO -  epoch: 584, all client loss: [0.5569390058517456, 0.46745359897613525], all pred client disparities: [0.009942680597305298, 0.00023831427097320557], all client disparities: [0.019202888011932373, 0.011769808828830719], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:15,378 - utils - INFO - stage1_gradient_single_runtime: 0.0025386810302734375
2023-09-28 23:27:15,380 - utils - INFO -  epoch: 585, all client loss: [0.5560895800590515, 0.4674268066883087], all pred client disparities: [0.010900884866714478, 0.0005428344011306763], all client disparities: [0.019202888011932373, 0.01212482899427414], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5867, 0.4133], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:15,619 - utils - INFO - stage1_gradient_single_runtime: 0.002264261245727539
2023-09-28 23:27:15,619 - utils - INFO -  epoch: 586, all client loss: [0.5561967492103577, 0.46732038259506226], all pred client disparities: [0.010705292224884033, 0.0004551112651824951], all client disparities: [0.019202888011932373, 0.01212482899427414], all client accs: [0.7481840252876282, 0.7766579389572144],  alpha_performance: tensor([0.5880, 0.4120], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:15,844 - utils - INFO - stage1_gradient_single_runtime: 0.0024394989013671875
2023-09-28 23:27:15,845 - utils - INFO -  epoch: 587, all client loss: [0.5563026666641235, 0.467215359210968], all pred client disparities: [0.010506749153137207, 0.00037026405334472656], all client disparities: [0.019202888011932373, 0.011477455496788025], all client accs: [0.7481840252876282, 0.7765957117080688],  alpha_performance: tensor([0.5892, 0.4108], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:16,076 - utils - INFO - stage1_gradient_single_runtime: 0.0022573471069335938
2023-09-28 23:27:16,077 - utils - INFO -  epoch: 588, all client loss: [0.5564072132110596, 0.46711164712905884], all pred client disparities: [0.010305076837539673, 0.00028824806213378906], all client disparities: [0.019202888011932373, 0.011550545692443848], all client accs: [0.7481840252876282, 0.7766890525817871],  alpha_performance: tensor([0.5904, 0.4096], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:16,300 - utils - INFO - stage1_gradient_single_runtime: 0.0022802352905273438
2023-09-28 23:27:16,301 - utils - INFO -  epoch: 589, all client loss: [0.5565105676651001, 0.4670090973377228], all pred client disparities: [0.010100305080413818, 0.00020898878574371338], all client disparities: [0.019202888011932373, 0.011769808828830719], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5917, 0.4083], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:16,527 - utils - INFO - stage1_gradient_single_runtime: 0.002063274383544922
2023-09-28 23:27:16,528 - utils - INFO -  epoch: 590, all client loss: [0.5566126704216003, 0.466907799243927], all pred client disparities: [0.009892463684082031, 0.00013238191604614258], all client disparities: [0.019202888011932373, 0.011769808828830719], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:16,710 - utils - INFO - stage1_gradient_single_runtime: 0.0022497177124023438
2023-09-28 23:27:16,711 - utils - INFO -  epoch: 591, all client loss: [0.5557671189308167, 0.46688130497932434], all pred client disparities: [0.010848701000213623, 0.00043508410453796387], all client disparities: [0.019202888011932373, 0.01212482899427414], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5872, 0.4128], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:16,934 - utils - INFO - stage1_gradient_single_runtime: 0.0022711753845214844
2023-09-28 23:27:16,935 - utils - INFO -  epoch: 592, all client loss: [0.555871844291687, 0.46677732467651367], all pred client disparities: [0.010655254125595093, 0.00034953653812408447], all client disparities: [0.019202888011932373, 0.012197919189929962], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5884, 0.4116], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:17,158 - utils - INFO - stage1_gradient_single_runtime: 0.002274751663208008
2023-09-28 23:27:17,159 - utils - INFO -  epoch: 593, all client loss: [0.5559753775596619, 0.4666746258735657], all pred client disparities: [0.010458946228027344, 0.00026679039001464844], all client disparities: [0.019202888011932373, 0.011550545692443848], all client accs: [0.7481840252876282, 0.7766890525817871],  alpha_performance: tensor([0.5896, 0.4104], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:17,382 - utils - INFO - stage1_gradient_single_runtime: 0.0025331974029541016
2023-09-28 23:27:17,384 - utils - INFO -  epoch: 594, all client loss: [0.5560776591300964, 0.46657320857048035], all pred client disparities: [0.010259509086608887, 0.00018678605556488037], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7766579389572144],  alpha_performance: tensor([0.5908, 0.4092], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:17,613 - utils - INFO - stage1_gradient_single_runtime: 0.002281665802001953
2023-09-28 23:27:17,614 - utils - INFO -  epoch: 595, all client loss: [0.5561786890029907, 0.4664729833602905], all pred client disparities: [0.010057061910629272, 0.00010944902896881104], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7766579389572144],  alpha_performance: tensor([0.5921, 0.4079], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:17,837 - utils - INFO - stage1_gradient_single_runtime: 0.0020661354064941406
2023-09-28 23:27:17,838 - utils - INFO -  epoch: 596, all client loss: [0.5562785267829895, 0.4663739502429962], all pred client disparities: [0.009851604700088501, 3.4734610380837694e-05], all client disparities: [0.019202888011932373, 0.011769808828830719], all client accs: [0.7481840252876282, 0.7767201662063599],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:18,071 - utils - INFO - stage1_gradient_single_runtime: 0.0022428035736083984
2023-09-28 23:27:18,071 - utils - INFO -  epoch: 597, all client loss: [0.5554372072219849, 0.4663476049900055], all pred client disparities: [0.010805070400238037, 0.00033582746982574463], all client disparities: [0.019202888011932373, 0.01212482899427414], all client accs: [0.7481840252876282, 0.7766268253326416],  alpha_performance: tensor([0.5876, 0.4124], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:18,301 - utils - INFO - stage1_gradient_single_runtime: 0.002259969711303711
2023-09-28 23:27:18,302 - utils - INFO -  epoch: 598, all client loss: [0.5555397868156433, 0.4662458896636963], all pred client disparities: [0.010613888502120972, 0.0002522021532058716], all client disparities: [0.019202888011932373, 0.012271009385585785], all client accs: [0.7481840252876282, 0.7767201662063599],  alpha_performance: tensor([0.5888, 0.4112], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:18,525 - utils - INFO - stage1_gradient_single_runtime: 0.0022842884063720703
2023-09-28 23:27:18,526 - utils - INFO -  epoch: 599, all client loss: [0.5556410551071167, 0.46614542603492737], all pred client disparities: [0.010419726371765137, 0.0001712888479232788], all client disparities: [0.019202888011932373, 0.011550545692443848], all client accs: [0.7481840252876282, 0.7766890525817871],  alpha_performance: tensor([0.5900, 0.4100], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:18,609 - utils - INFO - valid: True, epoch: 599, loss: [0.5882611274719238, 0.4649498462677002], accuracy: [0.7016574740409851, 0.7795652151107788], mean_accuracy:0.740611344575882,variance_accuracy:0.03895387053489685, disparity: [0.016666650772094727, 0.015570715069770813], mean_disparity:0.01611868292093277,variance_disparity:0.0005479678511619568, pred_disparity: [0.033511072397232056, 0.004489630460739136]
2023-09-28 23:27:18,735 - utils - INFO - global_valid: True, epoch: 599,  global_loss: 0.4663207530975342, global_accuracy: 0.8124115917592072,  global_disparity:0.01778516173362732, global_pred_disparity: 0.006227314472198486,
2023-09-28 23:27:18,954 - utils - INFO - stage1_gradient_single_runtime: 0.0022628307342529297
2023-09-28 23:27:18,955 - utils - INFO -  epoch: 600, all client loss: [0.5557410717010498, 0.46604618430137634], all pred client disparities: [0.010222762823104858, 9.305775165557861e-05], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7766579389572144],  alpha_performance: tensor([0.5912, 0.4088], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:19,177 - utils - INFO - stage1_gradient_single_runtime: 0.0022907257080078125
2023-09-28 23:27:19,178 - utils - INFO -  epoch: 601, all client loss: [0.5558399558067322, 0.4659481346607208], all pred client disparities: [0.010022729635238647, 1.7449265214963816e-05], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7767512798309326],  alpha_performance: tensor([0.5924, 0.4076], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:19,398 - utils - INFO - stage1_gradient_single_runtime: 0.0020656585693359375
2023-09-28 23:27:19,399 - utils - INFO -  epoch: 602, all client loss: [0.5559377074241638, 0.46585118770599365], all pred client disparities: [0.009819716215133667, 5.564093589782715e-05], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7767823934555054],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:19,624 - utils - INFO - stage1_gradient_single_runtime: 0.0022690296173095703
2023-09-28 23:27:19,625 - utils - INFO -  epoch: 603, all client loss: [0.5551010370254517, 0.4658249616622925], all pred client disparities: [0.010769546031951904, 0.00024402141571044922], all client disparities: [0.019202888011932373, 0.012197919189929962], all client accs: [0.7481840252876282, 0.7767201662063599],  alpha_performance: tensor([0.5880, 0.4120], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:19,845 - utils - INFO - stage1_gradient_single_runtime: 0.0022771358489990234
2023-09-28 23:27:19,846 - utils - INFO -  epoch: 604, all client loss: [0.5552014708518982, 0.4657253623008728], all pred client disparities: [0.010580688714981079, 0.0001620650291442871], all client disparities: [0.019202888011932373, 0.012197919189929962], all client accs: [0.7481840252876282, 0.7767512798309326],  alpha_performance: tensor([0.5892, 0.4108], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:20,072 - utils - INFO - stage1_gradient_single_runtime: 0.0022437572479248047
2023-09-28 23:27:20,074 - utils - INFO -  epoch: 605, all client loss: [0.5553007125854492, 0.46562692523002625], all pred client disparities: [0.01038902997970581, 8.280575275421143e-05], all client disparities: [0.019202888011932373, 0.011550545692443848], all client accs: [0.7481840252876282, 0.7766890525817871],  alpha_performance: tensor([0.5904, 0.4096], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:20,296 - utils - INFO - stage1_gradient_single_runtime: 0.002316713333129883
2023-09-28 23:27:20,297 - utils - INFO -  epoch: 606, all client loss: [0.5553986430168152, 0.4655297100543976], all pred client disparities: [0.01019442081451416, 6.124395440565422e-06], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7767512798309326],  alpha_performance: tensor([0.5915, 0.4085], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:20,522 - utils - INFO - stage1_gradient_single_runtime: 0.002053976058959961
2023-09-28 23:27:20,524 - utils - INFO -  epoch: 607, all client loss: [0.55549556016922, 0.46543365716934204], all pred client disparities: [0.009996920824050903, 6.799399852752686e-05], all client disparities: [0.019202888011932373, 0.011696718633174896], all client accs: [0.7481840252876282, 0.7767823934555054],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:20,757 - utils - INFO - stage1_gradient_single_runtime: 0.002568960189819336
2023-09-28 23:27:20,758 - utils - INFO -  epoch: 608, all client loss: [0.5546683669090271, 0.465407133102417], all pred client disparities: [0.01092827320098877, 0.0002323240041732788], all client disparities: [0.019202888011932373, 0.01212482899427414], all client accs: [0.7481840252876282, 0.7767201662063599],  alpha_performance: tensor([0.5872, 0.4128], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:20,989 - utils - INFO - stage1_gradient_single_runtime: 0.0025959014892578125
2023-09-28 23:27:20,991 - utils - INFO -  epoch: 609, all client loss: [0.554767906665802, 0.46530839800834656], all pred client disparities: [0.010744571685791016, 0.0001493692398071289], all client disparities: [0.019202888011932373, 0.01212482899427414], all client accs: [0.7481840252876282, 0.7767512798309326],  alpha_performance: tensor([0.5883, 0.4117], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:21,218 - utils - INFO - stage1_gradient_single_runtime: 0.002254009246826172
2023-09-28 23:27:21,218 - utils - INFO -  epoch: 610, all client loss: [0.5548662543296814, 0.465210884809494], all pred client disparities: [0.01055818796157837, 6.908178329467773e-05], all client disparities: [0.019202888011932373, 0.012197919189929962], all client accs: [0.7481840252876282, 0.7767512798309326],  alpha_performance: tensor([0.5894, 0.4106], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:21,439 - utils - INFO - stage1_gradient_single_runtime: 0.002227306365966797
2023-09-28 23:27:21,440 - utils - INFO -  epoch: 611, all client loss: [0.5549633502960205, 0.465114563703537], all pred client disparities: [0.010368973016738892, 8.627785064163618e-06], all client disparities: [0.019202888011932373, 0.011550545692443848], all client accs: [0.7481840252876282, 0.7767823934555054],  alpha_performance: tensor([0.4275, 0.5725], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:21,667 - utils - INFO - stage1_gradient_single_runtime: 0.0020482540130615234
2023-09-28 23:27:21,668 - utils - INFO -  epoch: 612, all client loss: [0.5549342036247253, 0.46514445543289185], all pred client disparities: [0.009964138269424438, 0.00023247301578521729], all client disparities: [0.019202888011932373, 0.011550545692443848], all client accs: [0.7481840252876282, 0.7766890525817871],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:21,893 - utils - INFO - stage1_gradient_single_runtime: 0.0022406578063964844
2023-09-28 23:27:21,894 - utils - INFO -  epoch: 613, all client loss: [0.554121732711792, 0.46511712670326233], all pred client disparities: [0.01088210940361023, 0.0005376487970352173], all client disparities: [0.019202888011932373, 0.005400769412517548], all client accs: [0.7481840252876282, 0.781417191028595],  alpha_performance: tensor([0.5895, 0.4105], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:22,116 - utils - INFO - stage1_gradient_single_runtime: 0.002256631851196289
2023-09-28 23:27:22,117 - utils - INFO -  epoch: 614, all client loss: [0.5542234182357788, 0.46501630544662476], all pred client disparities: [0.010700792074203491, 0.0004505962133407593], all client disparities: [0.019202888011932373, 0.00547385960817337], all client accs: [0.7481840252876282, 0.7815105319023132],  alpha_performance: tensor([0.5906, 0.4094], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:22,345 - utils - INFO - stage1_gradient_single_runtime: 0.003667593002319336
2023-09-28 23:27:22,347 - utils - INFO -  epoch: 615, all client loss: [0.5543238520622253, 0.46491673588752747], all pred client disparities: [0.010516762733459473, 0.00036625564098358154], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5917, 0.4083], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:22,606 - utils - INFO - stage1_gradient_single_runtime: 0.0037126541137695312
2023-09-28 23:27:22,607 - utils - INFO -  epoch: 616, all client loss: [0.554422914981842, 0.46481841802597046], all pred client disparities: [0.010330140590667725, 0.0002846270799636841], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5928, 0.4072], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:22,877 - utils - INFO - stage1_gradient_single_runtime: 0.00628352165222168
2023-09-28 23:27:22,878 - utils - INFO -  epoch: 617, all client loss: [0.554520845413208, 0.46472132205963135], all pred client disparities: [0.01014062762260437, 0.0002056211233139038], all client disparities: [0.019202888011932373, 0.01212482899427414], all client accs: [0.7481840252876282, 0.7767512798309326],  alpha_performance: tensor([0.5940, 0.4060], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:23,183 - utils - INFO - stage1_gradient_single_runtime: 0.0021941661834716797
2023-09-28 23:27:23,184 - utils - INFO -  epoch: 618, all client loss: [0.5546175837516785, 0.46462538838386536], all pred client disparities: [0.009948402643203735, 0.00012917816638946533], all client disparities: [0.019202888011932373, 0.011477455496788025], all client accs: [0.7481840252876282, 0.7768135070800781],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:23,417 - utils - INFO - stage1_gradient_single_runtime: 0.0027494430541992188
2023-09-28 23:27:23,418 - utils - INFO -  epoch: 619, all client loss: [0.5538091063499451, 0.46459826827049255], all pred client disparities: [0.010862678289413452, 0.00043278932571411133], all client disparities: [0.019202888011932373, 0.00547385960817337], all client accs: [0.7481840252876282, 0.7815105319023132],  alpha_performance: tensor([0.5897, 0.4103], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:23,504 - utils - INFO - valid: True, epoch: 619, loss: [0.5870934128761292, 0.46340614557266235], accuracy: [0.7016574740409851, 0.7840372920036316], mean_accuracy:0.7428473830223083,variance_accuracy:0.04118990898132324, disparity: [0.016666650772094727, 0.00857902318239212], mean_disparity:0.012622836977243423,variance_disparity:0.004043813794851303, pred_disparity: [0.032630592584609985, 0.004100576043128967]
2023-09-28 23:27:23,630 - utils - INFO - global_valid: True, epoch: 619,  global_loss: 0.4647812247276306, global_accuracy: 0.8128641374042578,  global_disparity:0.01096712052822113, global_pred_disparity: 0.005886852741241455,
2023-09-28 23:27:23,851 - utils - INFO - stage1_gradient_single_runtime: 0.0022974014282226562
2023-09-28 23:27:23,851 - utils - INFO -  epoch: 620, all client loss: [0.5539084672927856, 0.4644997715950012], all pred client disparities: [0.010683685541152954, 0.0003476440906524658], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5908, 0.4092], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:24,076 - utils - INFO - stage1_gradient_single_runtime: 0.0022585391998291016
2023-09-28 23:27:24,077 - utils - INFO -  epoch: 621, all client loss: [0.5540065169334412, 0.4644024968147278], all pred client disparities: [0.010502070188522339, 0.0002651512622833252], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5919, 0.4081], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:24,299 - utils - INFO - stage1_gradient_single_runtime: 0.002261638641357422
2023-09-28 23:27:24,300 - utils - INFO -  epoch: 622, all client loss: [0.5541033744812012, 0.46430638432502747], all pred client disparities: [0.010317862033843994, 0.0001852661371231079], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7815105319023132],  alpha_performance: tensor([0.5930, 0.4070], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:24,519 - utils - INFO - stage1_gradient_single_runtime: 0.002267122268676758
2023-09-28 23:27:24,520 - utils - INFO -  epoch: 623, all client loss: [0.5541990995407104, 0.46421149373054504], all pred client disparities: [0.010130882263183594, 0.00010794401168823242], all client disparities: [0.019202888011932373, 0.01212482899427414], all client accs: [0.7481840252876282, 0.7768756747245789],  alpha_performance: tensor([0.5941, 0.4059], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:24,744 - utils - INFO - stage1_gradient_single_runtime: 0.002080202102661133
2023-09-28 23:27:24,745 - utils - INFO -  epoch: 624, all client loss: [0.5542936325073242, 0.46411773562431335], all pred client disparities: [0.0099412202835083, 3.314018249511719e-05], all client disparities: [0.019202888011932373, 0.011404365301132202], all client accs: [0.7481840252876282, 0.7768756747245789],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:24,971 - utils - INFO - stage1_gradient_single_runtime: 0.0022461414337158203
2023-09-28 23:27:24,972 - utils - INFO -  epoch: 625, all client loss: [0.5534895062446594, 0.4640907347202301], all pred client disparities: [0.010850995779037476, 0.0003353804349899292], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5899, 0.4101], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:25,195 - utils - INFO - stage1_gradient_single_runtime: 0.00225830078125
2023-09-28 23:27:25,196 - utils - INFO -  epoch: 626, all client loss: [0.5535866022109985, 0.4639943838119507], all pred client disparities: [0.01067441701889038, 0.00025190412998199463], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5909, 0.4091], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:25,425 - utils - INFO - stage1_gradient_single_runtime: 0.002257823944091797
2023-09-28 23:27:25,426 - utils - INFO -  epoch: 627, all client loss: [0.5536825656890869, 0.4638992249965668], all pred client disparities: [0.010495305061340332, 0.0001710355281829834], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.781572699546814],  alpha_performance: tensor([0.5920, 0.4080], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:25,650 - utils - INFO - stage1_gradient_single_runtime: 0.0022554397583007812
2023-09-28 23:27:25,651 - utils - INFO -  epoch: 628, all client loss: [0.5537773370742798, 0.463805228471756], all pred client disparities: [0.010313600301742554, 9.268522262573242e-05], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7816349267959595],  alpha_performance: tensor([0.5931, 0.4069], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:25,876 - utils - INFO - stage1_gradient_single_runtime: 0.0022454261779785156
2023-09-28 23:27:25,877 - utils - INFO -  epoch: 629, all client loss: [0.5538710355758667, 0.4637123942375183], all pred client disparities: [0.010129183530807495, 1.683831214904785e-05], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7816660404205322],  alpha_performance: tensor([0.5941, 0.4059], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:26,103 - utils - INFO - stage1_gradient_single_runtime: 0.002068042755126953
2023-09-28 23:27:26,104 - utils - INFO -  epoch: 630, all client loss: [0.5539634227752686, 0.4636206328868866], all pred client disparities: [0.009942114353179932, 5.65946102142334e-05], all client disparities: [0.019202888011932373, 0.011477455496788025], all client accs: [0.7481840252876282, 0.7769379019737244],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:26,327 - utils - INFO - stage1_gradient_single_runtime: 0.0022492408752441406
2023-09-28 23:27:26,328 - utils - INFO -  epoch: 631, all client loss: [0.553164005279541, 0.4635936915874481], all pred client disparities: [0.010846555233001709, 0.0002444833517074585], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5900, 0.4100], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:26,550 - utils - INFO - stage1_gradient_single_runtime: 0.0022754669189453125
2023-09-28 23:27:26,551 - utils - INFO -  epoch: 632, all client loss: [0.5532591342926025, 0.46349936723709106], all pred client disparities: [0.010672509670257568, 0.0001624375581741333], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7816038131713867],  alpha_performance: tensor([0.5910, 0.4090], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:26,784 - utils - INFO - stage1_gradient_single_runtime: 0.0022253990173339844
2023-09-28 23:27:26,785 - utils - INFO -  epoch: 633, all client loss: [0.5533530712127686, 0.46340617537498474], all pred client disparities: [0.010495901107788086, 8.29547643661499e-05], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7816038131713867],  alpha_performance: tensor([0.5921, 0.4079], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:27,009 - utils - INFO - stage1_gradient_single_runtime: 0.00251007080078125
2023-09-28 23:27:27,010 - utils - INFO -  epoch: 634, all client loss: [0.5534459352493286, 0.4633141756057739], all pred client disparities: [0.010316818952560425, 5.9604644775390625e-06], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816349267959595],  alpha_performance: tensor([0.5931, 0.4069], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:27,233 - utils - INFO - stage1_gradient_single_runtime: 0.00223541259765625
2023-09-28 23:27:27,233 - utils - INFO -  epoch: 635, all client loss: [0.5535375475883484, 0.46322327852249146], all pred client disparities: [0.010135084390640259, 6.861984729766846e-05], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.4290, 0.5710], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:27,476 - utils - INFO - stage1_gradient_single_runtime: 0.0025758743286132812
2023-09-28 23:27:27,477 - utils - INFO -  epoch: 636, all client loss: [0.5535094141960144, 0.46325206756591797], all pred client disparities: [0.009738266468048096, 0.00016990303993225098], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:27,704 - utils - INFO - stage1_gradient_single_runtime: 0.002324819564819336
2023-09-28 23:27:27,705 - utils - INFO -  epoch: 637, all client loss: [0.5527200698852539, 0.46322470903396606], all pred client disparities: [0.010641932487487793, 0.00047400593757629395], all client disparities: [0.019202888011932373, 0.00547385960817337], all client accs: [0.7481840252876282, 0.781541645526886],  alpha_performance: tensor([0.5933, 0.4067], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:27,931 - utils - INFO - stage1_gradient_single_runtime: 0.0022554397583007812
2023-09-28 23:27:27,932 - utils - INFO -  epoch: 638, all client loss: [0.5528161525726318, 0.46312952041625977], all pred client disparities: [0.010467767715454102, 0.0003903508186340332], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7815105319023132],  alpha_performance: tensor([0.5943, 0.4057], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:28,160 - utils - INFO - stage1_gradient_single_runtime: 0.0022308826446533203
2023-09-28 23:27:28,161 - utils - INFO -  epoch: 639, all client loss: [0.5529109239578247, 0.4630354344844818], all pred client disparities: [0.010291039943695068, 0.00030925869941711426], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781572699546814],  alpha_performance: tensor([0.5954, 0.4046], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:28,247 - utils - INFO - valid: True, epoch: 639, loss: [0.5865200757980347, 0.4618418216705322], accuracy: [0.7016574740409851, 0.7842236161231995], mean_accuracy:0.7429405450820923,variance_accuracy:0.04128307104110718, disparity: [0.021212130784988403, 0.00857902318239212], mean_disparity:0.014895576983690262,variance_disparity:0.0063165538012981415, pred_disparity: [0.03285565972328186, 0.0041625648736953735]
2023-09-28 23:27:28,370 - utils - INFO - global_valid: True, epoch: 639,  global_loss: 0.46322792768478394, global_accuracy: 0.8131320516190169,  global_disparity:0.010823860764503479, global_pred_disparity: 0.005946800112724304,
2023-09-28 23:27:28,591 - utils - INFO - stage1_gradient_single_runtime: 0.0022330284118652344
2023-09-28 23:27:28,592 - utils - INFO -  epoch: 640, all client loss: [0.5530046224594116, 0.46294257044792175], all pred client disparities: [0.010111838579177856, 0.00023068487644195557], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816038131713867],  alpha_performance: tensor([0.5964, 0.4036], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:28,814 - utils - INFO - stage1_gradient_single_runtime: 0.002054929733276367
2023-09-28 23:27:28,814 - utils - INFO -  epoch: 641, all client loss: [0.553097128868103, 0.4628508388996124], all pred client disparities: [0.009930014610290527, 0.0001545548439025879], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816660404205322],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:29,038 - utils - INFO - stage1_gradient_single_runtime: 0.002250194549560547
2023-09-28 23:27:29,039 - utils - INFO -  epoch: 642, all client loss: [0.552316427230835, 0.4628232419490814], all pred client disparities: [0.010815560817718506, 0.0004592686891555786], all client disparities: [0.019202888011932373, 0.00547385960817337], all client accs: [0.7481840252876282, 0.781541645526886],  alpha_performance: tensor([0.5924, 0.4076], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:29,260 - utils - INFO - stage1_gradient_single_runtime: 0.002256631851196289
2023-09-28 23:27:29,261 - utils - INFO -  epoch: 643, all client loss: [0.5524114966392517, 0.46272894740104675], all pred client disparities: [0.010646313428878784, 0.0003746598958969116], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5933, 0.4067], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:29,482 - utils - INFO - stage1_gradient_single_runtime: 0.002246856689453125
2023-09-28 23:27:29,482 - utils - INFO -  epoch: 644, all client loss: [0.5525053143501282, 0.46263587474823], all pred client disparities: [0.010474562644958496, 0.0002925992012023926], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781572699546814],  alpha_performance: tensor([0.5943, 0.4057], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:29,702 - utils - INFO - stage1_gradient_single_runtime: 0.0022437572479248047
2023-09-28 23:27:29,703 - utils - INFO -  epoch: 645, all client loss: [0.5525980591773987, 0.4625439941883087], all pred client disparities: [0.010300397872924805, 0.0002130568027496338], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816038131713867],  alpha_performance: tensor([0.5953, 0.4047], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:29,926 - utils - INFO - stage1_gradient_single_runtime: 0.0022728443145751953
2023-09-28 23:27:29,927 - utils - INFO -  epoch: 646, all client loss: [0.5526895523071289, 0.4624532163143158], all pred client disparities: [0.010123759508132935, 0.00013597309589385986], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816660404205322],  alpha_performance: tensor([0.5963, 0.4037], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:30,147 - utils - INFO - stage1_gradient_single_runtime: 0.0020563602447509766
2023-09-28 23:27:30,148 - utils - INFO -  epoch: 647, all client loss: [0.552780032157898, 0.4623635411262512], all pred client disparities: [0.009944617748260498, 6.128847599029541e-05], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817593812942505],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:30,384 - utils - INFO - stage1_gradient_single_runtime: 0.002554655075073242
2023-09-28 23:27:30,385 - utils - INFO -  epoch: 648, all client loss: [0.5520037412643433, 0.4623360335826874], all pred client disparities: [0.01082456111907959, 0.000364840030670166], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7814794182777405],  alpha_performance: tensor([0.5923, 0.4077], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:30,611 - utils - INFO - stage1_gradient_single_runtime: 0.002221345901489258
2023-09-28 23:27:30,612 - utils - INFO -  epoch: 649, all client loss: [0.5520967245101929, 0.46224382519721985], all pred client disparities: [0.01065775752067566, 0.00028167665004730225], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781572699546814],  alpha_performance: tensor([0.5933, 0.4067], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:30,866 - utils - INFO - stage1_gradient_single_runtime: 0.0022993087768554688
2023-09-28 23:27:30,867 - utils - INFO -  epoch: 650, all client loss: [0.5521885752677917, 0.4621528387069702], all pred client disparities: [0.0104885995388031, 0.00020104646682739258], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816038131713867],  alpha_performance: tensor([0.5943, 0.4057], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:31,045 - utils - INFO - stage1_gradient_single_runtime: 0.0022344589233398438
2023-09-28 23:27:31,046 - utils - INFO -  epoch: 651, all client loss: [0.5522792339324951, 0.46206289529800415], all pred client disparities: [0.010317027568817139, 0.00012284517288208008], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5952, 0.4048], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:31,266 - utils - INFO - stage1_gradient_single_runtime: 0.002327442169189453
2023-09-28 23:27:31,267 - utils - INFO -  epoch: 652, all client loss: [0.5523687601089478, 0.4619740843772888], all pred client disparities: [0.010143101215362549, 4.70578670501709e-05], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5962, 0.4038], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:31,491 - utils - INFO - stage1_gradient_single_runtime: 0.002274751663208008
2023-09-28 23:27:31,492 - utils - INFO -  epoch: 653, all client loss: [0.5524572134017944, 0.46188634634017944], all pred client disparities: [0.009966641664505005, 2.64197624346707e-05], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817593812942505],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:31,719 - utils - INFO - stage1_gradient_single_runtime: 0.002258777618408203
2023-09-28 23:27:31,721 - utils - INFO -  epoch: 654, all client loss: [0.551685631275177, 0.4618588089942932], all pred client disparities: [0.010840237140655518, 0.0002761036157608032], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781572699546814],  alpha_performance: tensor([0.5923, 0.4077], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:31,948 - utils - INFO - stage1_gradient_single_runtime: 0.002244234085083008
2023-09-28 23:27:31,949 - utils - INFO -  epoch: 655, all client loss: [0.5517767071723938, 0.46176856756210327], all pred client disparities: [0.010676056146621704, 0.0001942366361618042], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781572699546814],  alpha_performance: tensor([0.5932, 0.4068], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:32,175 - utils - INFO - stage1_gradient_single_runtime: 0.0022878646850585938
2023-09-28 23:27:32,175 - utils - INFO -  epoch: 656, all client loss: [0.5518665909767151, 0.4616794288158417], all pred client disparities: [0.010509461164474487, 0.00011479854583740234], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5942, 0.4058], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:32,411 - utils - INFO - stage1_gradient_single_runtime: 0.0028638839721679688
2023-09-28 23:27:32,413 - utils - INFO -  epoch: 657, all client loss: [0.5519554018974304, 0.4615913927555084], all pred client disparities: [0.010340601205825806, 3.7774447264382616e-05], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5951, 0.4049], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:32,654 - utils - INFO - stage1_gradient_single_runtime: 0.0028243064880371094
2023-09-28 23:27:32,656 - utils - INFO -  epoch: 658, all client loss: [0.5520431399345398, 0.4615044593811035], all pred client disparities: [0.010169357061386108, 3.692507743835449e-05], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.4234, 0.5766], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:32,885 - utils - INFO - stage1_gradient_single_runtime: 0.0020868778228759766
2023-09-28 23:27:32,886 - utils - INFO -  epoch: 659, all client loss: [0.5520134568214417, 0.4615347981452942], all pred client disparities: [0.009783923625946045, 0.0002034902572631836], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:32,975 - utils - INFO - valid: True, epoch: 659, loss: [0.585417628288269, 0.4604082405567169], accuracy: [0.7016574740409851, 0.7840372920036316], mean_accuracy:0.7428473830223083,variance_accuracy:0.04118990898132324, disparity: [0.021212130784988403, 0.01004529744386673], mean_disparity:0.015628714114427567,variance_disparity:0.005583416670560837, pred_disparity: [0.03180897235870361, 0.003745630383491516]
2023-09-28 23:27:33,104 - utils - INFO - global_valid: True, epoch: 659,  global_loss: 0.4617980122566223, global_accuracy: 0.8137419417162693,  global_disparity:0.012228354811668396, global_pred_disparity: 0.005585148930549622,
2023-09-28 23:27:33,329 - utils - INFO - stage1_gradient_single_runtime: 0.002274751663208008
2023-09-28 23:27:33,330 - utils - INFO -  epoch: 660, all client loss: [0.5512518286705017, 0.46150681376457214], all pred client disparities: [0.010655790567398071, 0.0005090534687042236], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.781541645526886],  alpha_performance: tensor([0.5955, 0.4045], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:33,559 - utils - INFO - stage1_gradient_single_runtime: 0.002266407012939453
2023-09-28 23:27:33,560 - utils - INFO -  epoch: 661, all client loss: [0.5513437390327454, 0.46141573786735535], all pred client disparities: [0.010491520166397095, 0.00042544305324554443], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816038131713867],  alpha_performance: tensor([0.5964, 0.4036], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:33,787 - utils - INFO - stage1_gradient_single_runtime: 0.0022690296173095703
2023-09-28 23:27:33,788 - utils - INFO -  epoch: 662, all client loss: [0.5514344573020935, 0.4613257348537445], all pred client disparities: [0.010324984788894653, 0.0003443211317062378], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816660404205322],  alpha_performance: tensor([0.5974, 0.4026], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:34,012 - utils - INFO - stage1_gradient_single_runtime: 0.002251863479614258
2023-09-28 23:27:34,012 - utils - INFO -  epoch: 663, all client loss: [0.5515241026878357, 0.4612368941307068], all pred client disparities: [0.010156124830245972, 0.00026561319828033447], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5983, 0.4017], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:34,238 - utils - INFO - stage1_gradient_single_runtime: 0.002111673355102539
2023-09-28 23:27:34,239 - utils - INFO -  epoch: 664, all client loss: [0.5516126155853271, 0.461149126291275], all pred client disparities: [0.009984910488128662, 0.00018927454948425293], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:34,465 - utils - INFO - stage1_gradient_single_runtime: 0.0022554397583007812
2023-09-28 23:27:34,466 - utils - INFO -  epoch: 665, all client loss: [0.5508595705032349, 0.46112096309661865], all pred client disparities: [0.010838508605957031, 0.0004955381155014038], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7816349267959595],  alpha_performance: tensor([0.5945, 0.4055], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:34,693 - utils - INFO - stage1_gradient_single_runtime: 0.0022759437561035156
2023-09-28 23:27:34,694 - utils - INFO -  epoch: 666, all client loss: [0.5509505867958069, 0.4610307514667511], all pred client disparities: [0.010679006576538086, 0.00041091442108154297], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816038131713867],  alpha_performance: tensor([0.5953, 0.4047], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:34,919 - utils - INFO - stage1_gradient_single_runtime: 0.0022580623626708984
2023-09-28 23:27:34,920 - utils - INFO -  epoch: 667, all client loss: [0.5510403513908386, 0.46094170212745667], all pred client disparities: [0.010517328977584839, 0.000328749418258667], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816660404205322],  alpha_performance: tensor([0.5962, 0.4038], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:35,145 - utils - INFO - stage1_gradient_single_runtime: 0.0022492408752441406
2023-09-28 23:27:35,146 - utils - INFO -  epoch: 668, all client loss: [0.5511291027069092, 0.46085378527641296], all pred client disparities: [0.01035335659980774, 0.0002490133047103882], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5971, 0.4029], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:35,380 - utils - INFO - stage1_gradient_single_runtime: 0.0022749900817871094
2023-09-28 23:27:35,381 - utils - INFO -  epoch: 669, all client loss: [0.5512166619300842, 0.46076691150665283], all pred client disparities: [0.01018717885017395, 0.00017161667346954346], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5981, 0.4019], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:35,610 - utils - INFO - stage1_gradient_single_runtime: 0.0022394657135009766
2023-09-28 23:27:35,611 - utils - INFO -  epoch: 670, all client loss: [0.5513032078742981, 0.46068117022514343], all pred client disparities: [0.010018676519393921, 9.658932685852051e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5990, 0.4010], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:35,838 - utils - INFO - stage1_gradient_single_runtime: 0.0020682811737060547
2023-09-28 23:27:35,839 - utils - INFO -  epoch: 671, all client loss: [0.551388680934906, 0.4605964124202728], all pred client disparities: [0.009847909212112427, 2.378225326538086e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:36,067 - utils - INFO - stage1_gradient_single_runtime: 0.0022411346435546875
2023-09-28 23:27:36,068 - utils - INFO -  epoch: 672, all client loss: [0.5506360530853271, 0.46056848764419556], all pred client disparities: [0.010706692934036255, 0.00032736361026763916], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5952, 0.4048], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:36,296 - utils - INFO - stage1_gradient_single_runtime: 0.0022602081298828125
2023-09-28 23:27:36,297 - utils - INFO -  epoch: 673, all client loss: [0.5507239699363708, 0.4604812562465668], all pred client disparities: [0.010547518730163574, 0.0002463310956954956], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5961, 0.4039], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:36,531 - utils - INFO - stage1_gradient_single_runtime: 0.0022728443145751953
2023-09-28 23:27:36,532 - utils - INFO -  epoch: 674, all client loss: [0.5508108735084534, 0.4603951573371887], all pred client disparities: [0.010386258363723755, 0.00016763806343078613], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5970, 0.4030], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:36,756 - utils - INFO - stage1_gradient_single_runtime: 0.0024755001068115234
2023-09-28 23:27:36,759 - utils - INFO -  epoch: 675, all client loss: [0.5508967041969299, 0.46031007170677185], all pred client disparities: [0.01022273302078247, 9.12696123123169e-05], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817593812942505],  alpha_performance: tensor([0.5979, 0.4021], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:36,988 - utils - INFO - stage1_gradient_single_runtime: 0.0022513866424560547
2023-09-28 23:27:36,989 - utils - INFO -  epoch: 676, all client loss: [0.5509814620018005, 0.46022605895996094], all pred client disparities: [0.010057002305984497, 1.7181046132463962e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5988, 0.4012], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:37,217 - utils - INFO - stage1_gradient_single_runtime: 0.002142190933227539
2023-09-28 23:27:37,218 - utils - INFO -  epoch: 677, all client loss: [0.5510651469230652, 0.4601430892944336], all pred client disparities: [0.009888976812362671, 5.4717063903808594e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:37,447 - utils - INFO - stage1_gradient_single_runtime: 0.0022766590118408203
2023-09-28 23:27:37,448 - utils - INFO -  epoch: 678, all client loss: [0.5503175258636475, 0.4601150155067444], all pred client disparities: [0.0107402503490448, 0.00024813413619995117], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5951, 0.4049], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:37,671 - utils - INFO - stage1_gradient_single_runtime: 0.002242565155029297
2023-09-28 23:27:37,672 - utils - INFO -  epoch: 679, all client loss: [0.5504037737846375, 0.4600295126438141], all pred client disparities: [0.010583817958831787, 0.00016804039478302002], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5959, 0.4041], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:37,757 - utils - INFO - valid: True, epoch: 679, loss: [0.5849286913871765, 0.4588281214237213], accuracy: [0.7016574740409851, 0.7844099402427673], mean_accuracy:0.7430337071418762,variance_accuracy:0.04137623310089111, disparity: [0.025757580995559692, 0.00857902318239212], mean_disparity:0.017168302088975906,variance_disparity:0.008589278906583786, pred_disparity: [0.03146195411682129, 0.004047662019729614]
2023-09-28 23:27:37,884 - utils - INFO - global_valid: True, epoch: 679,  global_loss: 0.4602300226688385, global_accuracy: 0.8139910895993541,  global_disparity:0.010680586099624634, global_pred_disparity: 0.005902335047721863,
2023-09-28 23:27:38,104 - utils - INFO - stage1_gradient_single_runtime: 0.002299070358276367
2023-09-28 23:27:38,105 - utils - INFO -  epoch: 680, all client loss: [0.5504889488220215, 0.45994508266448975], all pred client disparities: [0.010425209999084473, 9.028613567352295e-05], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817593812942505],  alpha_performance: tensor([0.5968, 0.4032], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:38,330 - utils - INFO - stage1_gradient_single_runtime: 0.0022497177124023438
2023-09-28 23:27:38,331 - utils - INFO -  epoch: 681, all client loss: [0.5505730509757996, 0.45986172556877136], all pred client disparities: [0.010264396667480469, 1.4767058019060642e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5977, 0.4023], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:38,565 - utils - INFO - stage1_gradient_single_runtime: 0.0025260448455810547
2023-09-28 23:27:38,566 - utils - INFO -  epoch: 682, all client loss: [0.5506561994552612, 0.45977938175201416], all pred client disparities: [0.01010143756866455, 5.851686364621855e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.4204, 0.5796], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:38,805 - utils - INFO - stage1_gradient_single_runtime: 0.0020661354064941406
2023-09-28 23:27:38,805 - utils - INFO -  epoch: 683, all client loss: [0.5506258606910706, 0.45981037616729736], all pred client disparities: [0.009724318981170654, 0.00018225610256195068], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:39,029 - utils - INFO - stage1_gradient_single_runtime: 0.0022530555725097656
2023-09-28 23:27:39,030 - utils - INFO -  epoch: 684, all client loss: [0.5498883128166199, 0.4597819447517395], all pred client disparities: [0.010572761297225952, 0.0004882961511611938], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5982, 0.4018], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:39,251 - utils - INFO - stage1_gradient_single_runtime: 0.002247333526611328
2023-09-28 23:27:39,252 - utils - INFO -  epoch: 685, all client loss: [0.5499754548072815, 0.4596955180168152], all pred client disparities: [0.010416358709335327, 0.00040628015995025635], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5991, 0.4009], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:39,481 - utils - INFO - stage1_gradient_single_runtime: 0.002808809280395508
2023-09-28 23:27:39,483 - utils - INFO -  epoch: 686, all client loss: [0.5500615835189819, 0.4596101939678192], all pred client disparities: [0.010257899761199951, 0.0003266632556915283], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5999, 0.4001], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:39,713 - utils - INFO - stage1_gradient_single_runtime: 0.002268075942993164
2023-09-28 23:27:39,714 - utils - INFO -  epoch: 687, all client loss: [0.5501465201377869, 0.459526002407074], all pred client disparities: [0.010097324848175049, 0.000249326229095459], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.6008, 0.3992], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:39,940 - utils - INFO - stage1_gradient_single_runtime: 0.0022644996643066406
2023-09-28 23:27:39,940 - utils - INFO -  epoch: 688, all client loss: [0.5502305030822754, 0.45944276452064514], all pred client disparities: [0.00993451476097107, 0.0001742541790008545], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:40,167 - utils - INFO - stage1_gradient_single_runtime: 0.002254009246826172
2023-09-28 23:27:40,168 - utils - INFO -  epoch: 689, all client loss: [0.5495015382766724, 0.4594140946865082], all pred client disparities: [0.01076442003250122, 0.0004811137914657593], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5971, 0.4029], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:40,399 - utils - INFO - stage1_gradient_single_runtime: 0.002265453338623047
2023-09-28 23:27:40,399 - utils - INFO -  epoch: 690, all client loss: [0.5495878458023071, 0.45932847261428833], all pred client disparities: [0.010612636804580688, 0.00039793550968170166], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.781697154045105],  alpha_performance: tensor([0.5980, 0.4020], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:40,671 - utils - INFO - stage1_gradient_single_runtime: 0.0033261775970458984
2023-09-28 23:27:40,674 - utils - INFO -  epoch: 691, all client loss: [0.5496730804443359, 0.4592439532279968], all pred client disparities: [0.010458886623382568, 0.0003171265125274658], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817593812942505],  alpha_performance: tensor([0.5988, 0.4012], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:40,914 - utils - INFO - stage1_gradient_single_runtime: 0.002589702606201172
2023-09-28 23:27:40,915 - utils - INFO -  epoch: 692, all client loss: [0.5497573614120483, 0.4591605067253113], all pred client disparities: [0.01030305027961731, 0.00023858249187469482], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5996, 0.4004], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:41,145 - utils - INFO - stage1_gradient_single_runtime: 0.0023212432861328125
2023-09-28 23:27:41,148 - utils - INFO -  epoch: 693, all client loss: [0.5498404502868652, 0.4590780735015869], all pred client disparities: [0.010145187377929688, 0.00016231834888458252], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([0.6004, 0.3996], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:41,387 - utils - INFO - stage1_gradient_single_runtime: 0.0022122859954833984
2023-09-28 23:27:41,388 - utils - INFO -  epoch: 694, all client loss: [0.5499225854873657, 0.4589967131614685], all pred client disparities: [0.009985178709030151, 8.825957775115967e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:41,615 - utils - INFO - stage1_gradient_single_runtime: 0.002277374267578125
2023-09-28 23:27:41,616 - utils - INFO -  epoch: 695, all client loss: [0.5491982698440552, 0.458967924118042], all pred client disparities: [0.010807543992996216, 0.0003943145275115967], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7816660404205322],  alpha_performance: tensor([0.5969, 0.4031], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:41,843 - utils - INFO - stage1_gradient_single_runtime: 0.002262592315673828
2023-09-28 23:27:41,844 - utils - INFO -  epoch: 696, all client loss: [0.5492827892303467, 0.4588841199874878], all pred client disparities: [0.0106583833694458, 0.00031219422817230225], all client disparities: [0.019202888011932373, 0.005620032548904419], all client accs: [0.7481840252876282, 0.7817593812942505],  alpha_performance: tensor([0.5977, 0.4023], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:42,081 - utils - INFO - stage1_gradient_single_runtime: 0.002261638641357422
2023-09-28 23:27:42,081 - utils - INFO -  epoch: 697, all client loss: [0.549366295337677, 0.45880138874053955], all pred client disparities: [0.010507345199584961, 0.00023236870765686035], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5985, 0.4015], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:42,307 - utils - INFO - stage1_gradient_single_runtime: 0.0022687911987304688
2023-09-28 23:27:42,308 - utils - INFO -  epoch: 698, all client loss: [0.5494486689567566, 0.4587196707725525], all pred client disparities: [0.010354191064834595, 0.00015479326248168945], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817593812942505],  alpha_performance: tensor([0.5993, 0.4007], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:42,532 - utils - INFO - stage1_gradient_single_runtime: 0.0022614002227783203
2023-09-28 23:27:42,532 - utils - INFO -  epoch: 699, all client loss: [0.5495301485061646, 0.458638995885849], all pred client disparities: [0.010198980569839478, 7.939338684082031e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([0.6001, 0.3999], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:42,627 - utils - INFO - valid: True, epoch: 699, loss: [0.5843734741210938, 0.45743459463119507], accuracy: [0.7016574740409851, 0.7843478322029114], mean_accuracy:0.7430026531219482,variance_accuracy:0.041345179080963135, disparity: [0.025757580995559692, 0.008726947009563446], mean_disparity:0.01724226400256157,variance_disparity:0.008515316992998123, pred_disparity: [0.03137502074241638, 0.004068329930305481]
2023-09-28 23:27:42,754 - utils - INFO - global_valid: True, epoch: 699,  global_loss: 0.4588457942008972, global_accuracy: 0.8143729400161128,  global_disparity:0.010823860764503479, global_pred_disparity: 0.005933672189712524,
2023-09-28 23:27:42,977 - utils - INFO - stage1_gradient_single_runtime: 0.0022559165954589844
2023-09-28 23:27:42,977 - utils - INFO -  epoch: 700, all client loss: [0.5496105551719666, 0.4585592746734619], all pred client disparities: [0.010041773319244385, 6.198883056640625e-06], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([0.6009, 0.3991], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:43,203 - utils - INFO - stage1_gradient_single_runtime: 0.0020477771759033203
2023-09-28 23:27:43,203 - utils - INFO -  epoch: 701, all client loss: [0.5496900081634521, 0.4584805369377136], all pred client disparities: [0.009882420301437378, 6.489455699920654e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:43,429 - utils - INFO - stage1_gradient_single_runtime: 0.002280712127685547
2023-09-28 23:27:43,430 - utils - INFO -  epoch: 702, all client loss: [0.5489668250083923, 0.4584518373012543], all pred client disparities: [0.010707587003707886, 0.00023892521858215332], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7817904949188232],  alpha_performance: tensor([0.5974, 0.4026], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:43,657 - utils - INFO - stage1_gradient_single_runtime: 0.0022954940795898438
2023-09-28 23:27:43,657 - utils - INFO -  epoch: 703, all client loss: [0.5490487813949585, 0.4583706259727478], all pred client disparities: [0.010559141635894775, 0.00015982985496520996], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.7817282676696777],  alpha_performance: tensor([0.5982, 0.4018], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:43,883 - utils - INFO - stage1_gradient_single_runtime: 0.0022644996643066406
2023-09-28 23:27:43,884 - utils - INFO -  epoch: 704, all client loss: [0.5491296052932739, 0.4582904875278473], all pred client disparities: [0.010408759117126465, 8.292496204376221e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([0.5990, 0.4010], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:44,110 - utils - INFO - stage1_gradient_single_runtime: 0.0022993087768554688
2023-09-28 23:27:44,111 - utils - INFO -  epoch: 705, all client loss: [0.5492095351219177, 0.45821133255958557], all pred client disparities: [0.010256320238113403, 8.195638656616211e-06], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([0.5998, 0.4002], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:44,339 - utils - INFO - stage1_gradient_single_runtime: 0.0022623538970947266
2023-09-28 23:27:44,340 - utils - INFO -  epoch: 706, all client loss: [0.5492884516716003, 0.45813310146331787], all pred client disparities: [0.010101914405822754, 6.443262100219727e-05], all client disparities: [0.019202888011932373, 0.004899576306343079], all client accs: [0.7481840252876282, 0.781852662563324],  alpha_performance: tensor([0.4152, 0.5848], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:44,566 - utils - INFO - stage1_gradient_single_runtime: 0.002220630645751953
2023-09-28 23:27:44,567 - utils - INFO -  epoch: 707, all client loss: [0.5492568612098694, 0.45816537737846375], all pred client disparities: [0.00973328948020935, 0.0001779496669769287], all client disparities: [0.019202888011932373, 0.004826486110687256], all client accs: [0.7481840252876282, 0.7818837761878967],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:44,806 - utils - INFO - stage1_gradient_single_runtime: 0.002266407012939453
2023-09-28 23:27:44,807 - utils - INFO -  epoch: 708, all client loss: [0.5485438108444214, 0.45813626050949097], all pred client disparities: [0.010554581880569458, 0.0004850029945373535], all client disparities: [0.019202888011932373, 0.004826486110687256], all client accs: [0.7481840252876282, 0.7818215489387512],  alpha_performance: tensor([0.6005, 0.3995], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:45,040 - utils - INFO - stage1_gradient_single_runtime: 0.0022745132446289062
2023-09-28 23:27:45,041 - utils - INFO -  epoch: 709, all client loss: [0.548626720905304, 0.45805415511131287], all pred client disparities: [0.01040637493133545, 0.0004038810729980469], all client disparities: [0.019202888011932373, 0.004826486110687256], all client accs: [0.7481840252876282, 0.7818837761878967],  alpha_performance: tensor([0.6013, 0.3987], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:45,268 - utils - INFO - stage1_gradient_single_runtime: 0.002249479293823242
2023-09-28 23:27:45,268 - utils - INFO -  epoch: 710, all client loss: [0.5487084984779358, 0.45797306299209595], all pred client disparities: [0.010256201028823853, 0.0003249943256378174], all client disparities: [0.019202888011932373, 0.004826486110687256], all client accs: [0.7481840252876282, 0.7818837761878967],  alpha_performance: tensor([0.6020, 0.3980], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:45,446 - utils - INFO - stage1_gradient_single_runtime: 0.0023119449615478516
2023-09-28 23:27:45,446 - utils - INFO -  epoch: 711, all client loss: [0.5487892627716064, 0.4578930139541626], all pred client disparities: [0.01010403037071228, 0.0002482980489730835], all client disparities: [0.019202888011932373, 0.004826486110687256], all client accs: [0.7481840252876282, 0.7818837761878967],  alpha_performance: tensor([0.6028, 0.3972], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:45,721 - utils - INFO - stage1_gradient_single_runtime: 0.0024077892303466797
2023-09-28 23:27:45,723 - utils - INFO -  epoch: 712, all client loss: [0.5488690733909607, 0.45781394839286804], all pred client disparities: [0.009949862957000732, 0.00017379224300384521], all client disparities: [0.019202888011932373, 0.004826486110687256], all client accs: [0.7481840252876282, 0.7818837761878967],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:45,966 - utils - INFO - stage1_gradient_single_runtime: 0.0024254322052001953
2023-09-28 23:27:45,968 - utils - INFO -  epoch: 713, all client loss: [0.5481645464897156, 0.4577845633029938], all pred client disparities: [0.010752677917480469, 0.0004817545413970947], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7817904949188232],  alpha_performance: tensor([0.5994, 0.4006], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:46,198 - utils - INFO - stage1_gradient_single_runtime: 0.002282857894897461
2023-09-28 23:27:46,200 - utils - INFO -  epoch: 714, all client loss: [0.5482467412948608, 0.4577031135559082], all pred client disparities: [0.010609030723571777, 0.00039933621883392334], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7819460034370422],  alpha_performance: tensor([0.6001, 0.3999], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:46,490 - utils - INFO - stage1_gradient_single_runtime: 0.002276182174682617
2023-09-28 23:27:46,491 - utils - INFO -  epoch: 715, all client loss: [0.5483278036117554, 0.4576227366924286], all pred client disparities: [0.01046338677406311, 0.0003191530704498291], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7819460034370422],  alpha_performance: tensor([0.6009, 0.3991], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:46,716 - utils - INFO - stage1_gradient_single_runtime: 0.0022165775299072266
2023-09-28 23:27:46,718 - utils - INFO -  epoch: 716, all client loss: [0.5484079122543335, 0.45754340291023254], all pred client disparities: [0.010315865278244019, 0.00024117529392242432], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.781977117061615],  alpha_performance: tensor([0.6016, 0.3984], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:46,944 - utils - INFO - stage1_gradient_single_runtime: 0.0022618770599365234
2023-09-28 23:27:46,946 - utils - INFO -  epoch: 717, all client loss: [0.5484870076179504, 0.4574650228023529], all pred client disparities: [0.010166406631469727, 0.0001653730869293213], all client disparities: [0.019202888011932373, 0.005546942353248596], all client accs: [0.7481840252876282, 0.7819148898124695],  alpha_performance: tensor([0.6024, 0.3976], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:47,171 - utils - INFO - stage1_gradient_single_runtime: 0.0023055076599121094
2023-09-28 23:27:47,172 - utils - INFO -  epoch: 718, all client loss: [0.5485650897026062, 0.4573875963687897], all pred client disparities: [0.010015010833740234, 9.164214134216309e-05], all client disparities: [0.019202888011932373, 0.005693122744560242], all client accs: [0.7481840252876282, 0.7820082306861877],  alpha_performance: tensor([0.6032, 0.3968], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:47,399 - utils - INFO - stage1_gradient_single_runtime: 0.002046823501586914
2023-09-28 23:27:47,400 - utils - INFO -  epoch: 719, all client loss: [0.5486423373222351, 0.45731115341186523], all pred client disparities: [0.00986170768737793, 1.9997358322143555e-05], all client disparities: [0.019202888011932373, 0.005693122744560242], all client accs: [0.7481840252876282, 0.7820392847061157],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:47,486 - utils - INFO - valid: True, epoch: 719, loss: [0.5833438038825989, 0.4561595618724823], accuracy: [0.7016574740409851, 0.7844099402427673], mean_accuracy:0.7430337071418762,variance_accuracy:0.04137623310089111, disparity: [0.025757580995559692, 0.009897366166114807], mean_disparity:0.01782747358083725,variance_disparity:0.007930107414722443, pred_disparity: [0.030139416456222534, 0.003607526421546936]
2023-09-28 23:27:47,611 - utils - INFO - global_valid: True, epoch: 719,  global_loss: 0.4575735628604889, global_accuracy: 0.8150415337079908,  global_disparity:0.0119418203830719, global_pred_disparity: 0.0055359601974487305,
2023-09-28 23:27:47,835 - utils - INFO - stage1_gradient_single_runtime: 0.0022470951080322266
2023-09-28 23:27:47,836 - utils - INFO -  epoch: 720, all client loss: [0.5479389429092407, 0.45728179812431335], all pred client disparities: [0.010666579008102417, 0.0003258734941482544], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7819148898124695],  alpha_performance: tensor([0.5998, 0.4002], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:48,056 - utils - INFO - stage1_gradient_single_runtime: 0.002291440963745117
2023-09-28 23:27:48,057 - utils - INFO -  epoch: 721, all client loss: [0.5480185151100159, 0.4572029411792755], all pred client disparities: [0.010523557662963867, 0.0002463608980178833], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7819460034370422],  alpha_performance: tensor([0.6005, 0.3995], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:48,281 - utils - INFO - stage1_gradient_single_runtime: 0.0022704601287841797
2023-09-28 23:27:48,281 - utils - INFO -  epoch: 722, all client loss: [0.5480970740318298, 0.45712506771087646], all pred client disparities: [0.010378807783126831, 0.0001690387725830078], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.781977117061615],  alpha_performance: tensor([0.6013, 0.3987], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:48,506 - utils - INFO - stage1_gradient_single_runtime: 0.0022423267364501953
2023-09-28 23:27:48,506 - utils - INFO -  epoch: 723, all client loss: [0.5481746196746826, 0.4570482075214386], all pred client disparities: [0.010232120752334595, 9.381771087646484e-05], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820703983306885],  alpha_performance: tensor([0.6020, 0.3980], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:48,728 - utils - INFO - stage1_gradient_single_runtime: 0.0022678375244140625
2023-09-28 23:27:48,729 - utils - INFO -  epoch: 724, all client loss: [0.5482512712478638, 0.45697227120399475], all pred client disparities: [0.010083496570587158, 2.065300941467285e-05], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7821015119552612],  alpha_performance: tensor([0.6028, 0.3972], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:48,950 - utils - INFO - stage1_gradient_single_runtime: 0.0020606517791748047
2023-09-28 23:27:48,951 - utils - INFO -  epoch: 725, all client loss: [0.5483270287513733, 0.4568972587585449], all pred client disparities: [0.00993296504020691, 5.0440434279153123e-05], all client disparities: [0.019202888011932373, 0.005693122744560242], all client accs: [0.7481840252876282, 0.782132625579834],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:49,175 - utils - INFO - stage1_gradient_single_runtime: 0.0022454261779785156
2023-09-28 23:27:49,176 - utils - INFO -  epoch: 726, all client loss: [0.5476289391517639, 0.45686769485473633], all pred client disparities: [0.010728836059570312, 0.0002549886703491211], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7819148898124695],  alpha_performance: tensor([0.5995, 0.4005], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:49,408 - utils - INFO - stage1_gradient_single_runtime: 0.0022652149200439453
2023-09-28 23:27:49,409 - utils - INFO -  epoch: 727, all client loss: [0.5477069616317749, 0.45679035782814026], all pred client disparities: [0.010588526725769043, 0.0001760721206665039], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820082306861877],  alpha_performance: tensor([0.6002, 0.3998], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:49,631 - utils - INFO - stage1_gradient_single_runtime: 0.0022165775299072266
2023-09-28 23:27:49,633 - utils - INFO -  epoch: 728, all client loss: [0.547784149646759, 0.4567139148712158], all pred client disparities: [0.0104464590549469, 9.927153587341309e-05], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820392847061157],  alpha_performance: tensor([0.6009, 0.3991], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:49,863 - utils - INFO - stage1_gradient_single_runtime: 0.0022766590118408203
2023-09-28 23:27:49,865 - utils - INFO -  epoch: 729, all client loss: [0.5478602051734924, 0.45663848519325256], all pred client disparities: [0.010302484035491943, 2.4557113647460938e-05], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7821015119552612],  alpha_performance: tensor([0.6016, 0.3984], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:50,108 - utils - INFO - stage1_gradient_single_runtime: 0.002284526824951172
2023-09-28 23:27:50,109 - utils - INFO -  epoch: 730, all client loss: [0.5479355454444885, 0.45656391978263855], all pred client disparities: [0.010156750679016113, 4.8160552978515625e-05], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7821948528289795],  alpha_performance: tensor([0.4081, 0.5919], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:50,331 - utils - INFO - stage1_gradient_single_runtime: 0.002101898193359375
2023-09-28 23:27:50,332 - utils - INFO -  epoch: 731, all client loss: [0.5479022264480591, 0.4565978944301605], all pred client disparities: [0.009796500205993652, 0.0001970231533050537], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7821015119552612],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:50,564 - utils - INFO - stage1_gradient_single_runtime: 0.002268075942993164
2023-09-28 23:27:50,564 - utils - INFO -  epoch: 732, all client loss: [0.5472142100334167, 0.4565679728984833], all pred client disparities: [0.010588109493255615, 0.0005056858062744141], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7819148898124695],  alpha_performance: tensor([0.6025, 0.3975], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:50,798 - utils - INFO - stage1_gradient_single_runtime: 0.0022940635681152344
2023-09-28 23:27:50,798 - utils - INFO -  epoch: 733, all client loss: [0.5472932457923889, 0.456489622592926], all pred client disparities: [0.010448038578033447, 0.00042463839054107666], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7819460034370422],  alpha_performance: tensor([0.6032, 0.3968], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:51,025 - utils - INFO - stage1_gradient_single_runtime: 0.0022666454315185547
2023-09-28 23:27:51,026 - utils - INFO -  epoch: 734, all client loss: [0.5473713278770447, 0.45641228556632996], all pred client disparities: [0.01030626893043518, 0.0003457963466644287], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820392847061157],  alpha_performance: tensor([0.6039, 0.3961], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:51,261 - utils - INFO - stage1_gradient_single_runtime: 0.0022766590118408203
2023-09-28 23:27:51,262 - utils - INFO -  epoch: 735, all client loss: [0.5474483370780945, 0.4563359022140503], all pred client disparities: [0.010162591934204102, 0.0002690255641937256], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820392847061157],  alpha_performance: tensor([0.6046, 0.3954], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:51,486 - utils - INFO - stage1_gradient_single_runtime: 0.0022575855255126953
2023-09-28 23:27:51,487 - utils - INFO -  epoch: 736, all client loss: [0.5475245118141174, 0.4562605023384094], all pred client disparities: [0.01001712679862976, 0.00019435584545135498], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820703983306885],  alpha_performance: tensor([0.6053, 0.3947], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:51,710 - utils - INFO - stage1_gradient_single_runtime: 0.0020973682403564453
2023-09-28 23:27:51,710 - utils - INFO -  epoch: 737, all client loss: [0.5475996732711792, 0.4561859965324402], all pred client disparities: [0.00986984372138977, 0.00012171268463134766], all client disparities: [0.019202888011932373, 0.006632857024669647], all client accs: [0.7481840252876282, 0.782257080078125],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:51,933 - utils - INFO - stage1_gradient_single_runtime: 0.0023467540740966797
2023-09-28 23:27:51,934 - utils - INFO -  epoch: 738, all client loss: [0.5469164848327637, 0.4561559557914734], all pred client disparities: [0.010652780532836914, 0.0004298686981201172], all client disparities: [0.019202888011932373, 0.006987869739532471], all client accs: [0.7481840252876282, 0.7819148898124695],  alpha_performance: tensor([0.6021, 0.3979], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:52,157 - utils - INFO - stage1_gradient_single_runtime: 0.002265453338623047
2023-09-28 23:27:52,158 - utils - INFO -  epoch: 739, all client loss: [0.5469940304756165, 0.4560791254043579], all pred client disparities: [0.010515391826629639, 0.00034956634044647217], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820082306861877],  alpha_performance: tensor([0.6028, 0.3972], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:52,255 - utils - INFO - valid: True, epoch: 739, loss: [0.5828028321266174, 0.4548741281032562], accuracy: [0.7016574740409851, 0.7846583724021912], mean_accuracy:0.7431579232215881,variance_accuracy:0.04150044918060303, disparity: [0.025757580995559692, 0.010193221271038055], mean_disparity:0.017975401133298874,variance_disparity:0.0077821798622608185, pred_disparity: [0.029887914657592773, 0.0035949498414993286]
2023-09-28 23:27:52,381 - utils - INFO - global_valid: True, epoch: 739,  global_loss: 0.4562963843345642, global_accuracy: 0.8154323543267424,  global_disparity:0.012228354811668396, global_pred_disparity: 0.005540907382965088,
2023-09-28 23:27:52,600 - utils - INFO - stage1_gradient_single_runtime: 0.0022554397583007812
2023-09-28 23:27:52,601 - utils - INFO -  epoch: 740, all client loss: [0.5470705628395081, 0.4560032784938812], all pred client disparities: [0.010376185178756714, 0.00027133524417877197], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820392847061157],  alpha_performance: tensor([0.6035, 0.3965], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:52,825 - utils - INFO - stage1_gradient_single_runtime: 0.0022766590118408203
2023-09-28 23:27:52,825 - utils - INFO -  epoch: 741, all client loss: [0.5471461415290833, 0.4559284448623657], all pred client disparities: [0.010235249996185303, 0.00019519031047821045], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820703983306885],  alpha_performance: tensor([0.6041, 0.3959], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:53,059 - utils - INFO - stage1_gradient_single_runtime: 0.002624034881591797
2023-09-28 23:27:53,060 - utils - INFO -  epoch: 742, all client loss: [0.547220766544342, 0.45585447549819946], all pred client disparities: [0.01009252667427063, 0.00012108683586120605], all client disparities: [0.019202888011932373, 0.006632857024669647], all client accs: [0.7481840252876282, 0.7821637392044067],  alpha_performance: tensor([0.6048, 0.3952], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:53,284 - utils - INFO - stage1_gradient_single_runtime: 0.0020859241485595703
2023-09-28 23:27:53,285 - utils - INFO -  epoch: 743, all client loss: [0.5472944974899292, 0.4557814300060272], all pred client disparities: [0.009948015213012695, 4.898012048215605e-05], all client disparities: [0.019202888011932373, 0.006204746663570404], all client accs: [0.7481840252876282, 0.782132625579834],  alpha_performance: tensor([1., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:53,508 - utils - INFO - stage1_gradient_single_runtime: 0.0022351741790771484
2023-09-28 23:27:53,508 - utils - INFO -  epoch: 744, all client loss: [0.5466163754463196, 0.4557511806488037], all pred client disparities: [0.010721921920776367, 0.0003567636013031006], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820082306861877],  alpha_performance: tensor([0.6017, 0.3983], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:53,738 - utils - INFO - stage1_gradient_single_runtime: 0.002237081527709961
2023-09-28 23:27:53,739 - utils - INFO -  epoch: 745, all client loss: [0.5466923713684082, 0.4556758403778076], all pred client disparities: [0.010587155818939209, 0.00027698278427124023], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820392847061157],  alpha_performance: tensor([0.6024, 0.3976], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:53,974 - utils - INFO - stage1_gradient_single_runtime: 0.002249479293823242
2023-09-28 23:27:53,975 - utils - INFO -  epoch: 746, all client loss: [0.54676753282547, 0.45560145378112793], all pred client disparities: [0.010450690984725952, 0.0001993030309677124], all client disparities: [0.019202888011932373, 0.007134050130844116], all client accs: [0.7481840252876282, 0.7820703983306885],  alpha_performance: tensor([0.6030, 0.3970], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:54,207 - utils - INFO - stage1_gradient_single_runtime: 0.0022287368774414062
2023-09-28 23:27:54,208 - utils - INFO -  epoch: 747, all client loss: [0.5468416810035706, 0.45552799105644226], all pred client disparities: [0.010312438011169434, 0.0001236647367477417], all client disparities: [0.019202888011932373, 0.006131656467914581], all client accs: [0.7481840252876282, 0.7820703983306885],  alpha_performance: tensor([0.6037, 0.3963], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:54,431 - utils - INFO - stage1_gradient_single_runtime: 0.0022382736206054688
2023-09-28 23:27:54,433 - utils - INFO -  epoch: 748, all client loss: [0.5469148755073547, 0.45545539259910583], all pred client disparities: [0.010172396898269653, 5.002320176572539e-05], all client disparities: [0.019202888011932373, 0.006131656467914581], all client accs: [0.7481840252876282, 0.7821637392044067],  alpha_performance: tensor([0.6044, 0.3956], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:54,663 - utils - INFO - stage1_gradient_single_runtime: 0.002824068069458008
2023-09-28 23:27:54,664 - utils - INFO -  epoch: 749, all client loss: [0.5469872355461121, 0.4553837776184082], all pred client disparities: [0.01003071665763855, 2.1636486053466797e-05], all client disparities: [0.019202888011932373, 0.006204746663570404], all client accs: [0.7481840252876282, 0.782132625579834],  alpha_performance: tensor([0.4066, 0.5934], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:54,667 - utils - INFO - stage1_runtime: 180.90753436088562
2023-09-28 23:27:54,927 - utils - INFO - stage2_gradient_single_runtime: 0.0073773860931396484
2023-09-28 23:27:54,930 - utils - INFO - 1, epoch: 750, all client loss: [0.5469538569450378, 0.45541781187057495], all pred client disparities: [0.009673893451690674, 0.0002238452434539795], all client disparities: [0.019202888011932373, 0.006131656467914581], all client accs: [0.7481840252876282, 0.7821637392044067],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:55,170 - utils - INFO - stage2_gradient_single_runtime: 0.006292819976806641
2023-09-28 23:27:55,174 - utils - INFO - 1, epoch: 751, all client loss: [0.5459414720535278, 0.45580390095710754], all pred client disparities: [0.010505199432373047, 0.0007562190294265747], all client disparities: [0.019202888011932373, 0.005818456411361694], all client accs: [0.7481840252876282, 0.7821015119552612],  alphas:tensor([0.5364, 0.0000, 0.0000, 0.4636], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:55,404 - utils - INFO - stage2_gradient_single_runtime: 0.006339073181152344
2023-09-28 23:27:55,409 - utils - INFO - 1, epoch: 752, all client loss: [0.5459597706794739, 0.4559434652328491], all pred client disparities: [0.010253727436065674, 0.0007262974977493286], all client disparities: [0.019202888011932373, 0.005818456411361694], all client accs: [0.7481840252876282, 0.7821015119552612],  alphas:tensor([0.5360, 0.0000, 0.0000, 0.4640], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:55,638 - utils - INFO - stage2_gradient_single_runtime: 0.00628209114074707
2023-09-28 23:27:55,641 - utils - INFO - 1, epoch: 753, all client loss: [0.54598468542099, 0.45608198642730713], all pred client disparities: [0.009993374347686768, 0.0006943345069885254], all client disparities: [0.019202888011932373, 0.005745366215705872], all client accs: [0.7481840252876282, 0.7820392847061157],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:55,869 - utils - INFO - stage2_gradient_single_runtime: 0.006283283233642578
2023-09-28 23:27:55,872 - utils - INFO - 1, epoch: 754, all client loss: [0.5450004935264587, 0.4564822316169739], all pred client disparities: [0.010794997215270996, 0.0012549161911010742], all client disparities: [0.017391294240951538, 0.007614396512508392], all client accs: [0.7506053447723389, 0.7818215489387512],  alphas:tensor([0.5341, 0.0000, 0.0000, 0.4659], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:56,106 - utils - INFO - stage2_gradient_single_runtime: 0.006508350372314453
2023-09-28 23:27:56,111 - utils - INFO - 1, epoch: 755, all client loss: [0.5450277924537659, 0.4566175043582916], all pred client disparities: [0.01056024432182312, 0.0012150108814239502], all client disparities: [0.017391294240951538, 0.007614396512508392], all client accs: [0.7506053447723389, 0.7818215489387512],  alphas:tensor([0.5337, 0.0000, 0.0000, 0.4663], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:56,348 - utils - INFO - stage2_gradient_single_runtime: 0.006314754486083984
2023-09-28 23:27:56,353 - utils - INFO - 1, epoch: 756, all client loss: [0.5450609922409058, 0.4567517638206482], all pred client disparities: [0.010317087173461914, 0.0011731982231140137], all client disparities: [0.017391294240951538, 0.008188672363758087], all client accs: [0.7506053447723389, 0.7819148898124695],  alphas:tensor([0.5334, 0.0000, 0.0000, 0.4666], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:56,597 - utils - INFO - stage2_gradient_single_runtime: 0.006296396255493164
2023-09-28 23:27:56,602 - utils - INFO - 1, epoch: 757, all client loss: [0.5451001524925232, 0.4568850100040436], all pred client disparities: [0.0100649893283844, 0.0011295974254608154], all client disparities: [0.017391294240951538, 0.008188672363758087], all client accs: [0.7506053447723389, 0.7818837761878967],  alphas:tensor([0.5330, 0.0000, 0.0000, 0.4670], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:56,848 - utils - INFO - stage2_gradient_single_runtime: 0.0063097476959228516
2023-09-28 23:27:56,854 - utils - INFO - 1, epoch: 758, all client loss: [0.5451453924179077, 0.45701733231544495], all pred client disparities: [0.009803295135498047, 0.0010841339826583862], all client disparities: [0.017391294240951538, 0.008188672363758087], all client accs: [0.7506053447723389, 0.7819148898124695],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:57,102 - utils - INFO - stage2_gradient_single_runtime: 0.006334066390991211
2023-09-28 23:27:57,109 - utils - INFO - 1, epoch: 759, all client loss: [0.5441878437995911, 0.4574328064918518], all pred client disparities: [0.010608583688735962, 0.0016663968563079834], all client disparities: [0.013768106698989868, 0.009911522269248962], all client accs: [0.7530266642570496, 0.7817282676696777],  alphas:tensor([0.5314, 0.0000, 0.0000, 0.4686], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:57,191 - utils - INFO - valid: True, epoch: 759, loss: [0.5816441774368286, 0.45654669404029846], accuracy: [0.7016574740409851, 0.7833540439605713], mean_accuracy:0.7425057590007782,variance_accuracy:0.04084828495979309, disparity: [0.03030303120613098, 0.010328143835067749], mean_disparity:0.020315587520599365,variance_disparity:0.009987443685531616, pred_disparity: [0.028006315231323242, 0.002320587635040283]
2023-09-28 23:27:57,323 - utils - INFO - global_valid: True, epoch: 759,  global_loss: 0.45793747901916504, global_accuracy: 0.8150590141016174,  global_disparity:0.01220017671585083, global_pred_disparity: 0.0043742358684539795,
2023-09-28 23:27:57,580 - utils - INFO - stage2_gradient_single_runtime: 0.006790637969970703
2023-09-28 23:27:57,586 - utils - INFO - 1, epoch: 760, all client loss: [0.5442338585853577, 0.4575621485710144], all pred client disparities: [0.010371923446655273, 0.0016132445307448506], all client disparities: [0.013768106698989868, 0.009911522269248962], all client accs: [0.7530266642570496, 0.7817282676696777],  alphas:tensor([0.5312, 0.0000, 0.0000, 0.4688], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:57,835 - utils - INFO - stage2_gradient_single_runtime: 0.006292104721069336
2023-09-28 23:27:57,840 - utils - INFO - 1, epoch: 761, all client loss: [0.5442854762077332, 0.457690566778183], all pred client disparities: [0.010126322507858276, 0.0015584379434585571], all client disparities: [0.013768106698989868, 0.007896319031715393], all client accs: [0.7530266642570496, 0.7816038131713867],  alphas:tensor([0.5309, 0.0000, 0.0000, 0.4691], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:58,081 - utils - INFO - stage2_gradient_single_runtime: 0.006273746490478516
2023-09-28 23:27:58,086 - utils - INFO - 1, epoch: 762, all client loss: [0.5443426966667175, 0.4578181505203247], all pred client disparities: [0.009871125221252441, 0.0015019923448562622], all client disparities: [0.013768106698989868, 0.007969409227371216], all client accs: [0.7530266642570496, 0.781697154045105],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:58,324 - utils - INFO - stage2_gradient_single_runtime: 0.0062792301177978516
2023-09-28 23:27:58,329 - utils - INFO - 1, epoch: 763, all client loss: [0.5434098839759827, 0.45824772119522095], all pred client disparities: [0.010663777589797974, 0.002106711268424988], all client disparities: [0.02499997615814209, 0.010840825736522675], all client accs: [0.7602905631065369, 0.781697154045105],  alphas:tensor([0.5209, 0.0000, 0.0089, 0.4701], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:58,569 - utils - INFO - stage2_gradient_single_runtime: 0.006306648254394531
2023-09-28 23:27:58,574 - utils - INFO - 1, epoch: 764, all client loss: [0.5434693694114685, 0.4583684504032135], all pred client disparities: [0.010432392358779907, 0.0020406991243362427], all client disparities: [0.02499997615814209, 0.010840825736522675], all client accs: [0.7602905631065369, 0.781697154045105],  alphas:tensor([0.5190, 0.0000, 0.0107, 0.4703], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:58,810 - utils - INFO - stage2_gradient_single_runtime: 0.0063018798828125
2023-09-28 23:27:58,813 - utils - INFO - 1, epoch: 765, all client loss: [0.5435346364974976, 0.4584876298904419], all pred client disparities: [0.01019170880317688, 0.0019727498292922974], all client disparities: [0.02499997615814209, 0.01098700612783432], all client accs: [0.7602905631065369, 0.7816660404205322],  alphas:tensor([0.5172, 0.0000, 0.0124, 0.4704], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:59,053 - utils - INFO - stage2_gradient_single_runtime: 0.006258487701416016
2023-09-28 23:27:59,058 - utils - INFO - 1, epoch: 766, all client loss: [0.5436055064201355, 0.45860517024993896], all pred client disparities: [0.009941399097442627, 0.0019030123949050903], all client disparities: [0.02499997615814209, 0.01026654988527298], all client accs: [0.7602905631065369, 0.7816349267959595],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:59,295 - utils - INFO - stage2_gradient_single_runtime: 0.006242036819458008
2023-09-28 23:27:59,299 - utils - INFO - 1, epoch: 767, all client loss: [0.5426957607269287, 0.4590483009815216], all pred client disparities: [0.010721147060394287, 0.002528280019760132], all client disparities: [0.023188382387161255, 0.007238730788230896], all client accs: [0.7627118825912476, 0.7848077416419983],  alphas:tensor([0.5041, 0.0000, 0.0251, 0.4708], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:59,587 - utils - INFO - stage2_gradient_single_runtime: 0.006325960159301758
2023-09-28 23:27:59,590 - utils - INFO - 1, epoch: 768, all client loss: [0.5427685976028442, 0.45915743708610535], all pred client disparities: [0.01049390435218811, 0.002448365092277527], all client disparities: [0.023188382387161255, 0.007238730788230896], all client accs: [0.7627118825912476, 0.7848077416419983],  alphas:tensor([0.5027, 0.0000, 0.0264, 0.4709], device='cuda:0', dtype=torch.float64)
2023-09-28 23:27:59,883 - utils - INFO - stage2_gradient_single_runtime: 0.0063419342041015625
2023-09-28 23:27:59,888 - utils - INFO - 1, epoch: 769, all client loss: [0.5428466200828552, 0.45926523208618164], all pred client disparities: [0.010257482528686523, 0.0023669004440307617], all client disparities: [0.023188382387161255, 0.014203011989593506], all client accs: [0.7627118825912476, 0.7821637392044067],  alphas:tensor([0.5014, 0.0000, 0.0276, 0.4710], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:00,172 - utils - INFO - stage2_gradient_single_runtime: 0.010594844818115234
2023-09-28 23:28:00,177 - utils - INFO - 1, epoch: 770, all client loss: [0.5429297089576721, 0.45937180519104004], all pred client disparities: [0.010011404752731323, 0.002283945679664612], all client disparities: [0.023188382387161255, 0.012417502701282501], all client accs: [0.7627118825912476, 0.7818837761878967],  alphas:tensor([0.5003, 0.0000, 0.0287, 0.4710], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:00,367 - utils - INFO - stage2_gradient_single_runtime: 0.006246328353881836
2023-09-28 23:28:00,372 - utils - INFO - 1, epoch: 771, all client loss: [0.5430181622505188, 0.45947715640068054], all pred client disparities: [0.00975489616394043, 0.0021995753049850464], all client disparities: [0.023188382387161255, 0.011770136654376984], all client accs: [0.7627118825912476, 0.7818215489387512],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:00,657 - utils - INFO - stage2_gradient_single_runtime: 0.006336212158203125
2023-09-28 23:28:00,662 - utils - INFO - 1, epoch: 772, all client loss: [0.542128324508667, 0.4599331021308899], all pred client disparities: [0.010538369417190552, 0.0028397738933563232], all client disparities: [0.023188382387161255, 0.006873294711112976], all client accs: [0.7602905631065369, 0.7842167615890503],  alphas:tensor([0.4912, 0.0000, 0.0377, 0.4711], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:00,955 - utils - INFO - stage2_gradient_single_runtime: 0.006330251693725586
2023-09-28 23:28:00,958 - utils - INFO - 1, epoch: 773, all client loss: [0.5422161817550659, 0.4600319564342499], all pred client disparities: [0.01030537486076355, 0.002746596932411194], all client disparities: [0.023188382387161255, 0.007092565298080444], all client accs: [0.7602905631065369, 0.7847455739974976],  alphas:tensor([0.4902, 0.0000, 0.0386, 0.4712], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:01,207 - utils - INFO - stage2_gradient_single_runtime: 0.010468244552612305
2023-09-28 23:28:01,212 - utils - INFO - 1, epoch: 774, all client loss: [0.5423088669776917, 0.4601297080516815], all pred client disparities: [0.010062634944915771, 0.0026521533727645874], all client disparities: [0.023188382387161255, 0.007019475102424622], all client accs: [0.7602905631065369, 0.7847766280174255],  alphas:tensor([0.4893, 0.0000, 0.0395, 0.4713], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:01,466 - utils - INFO - stage2_gradient_single_runtime: 0.0062711238861083984
2023-09-28 23:28:01,469 - utils - INFO - 1, epoch: 775, all client loss: [0.5424064993858337, 0.4602263569831848], all pred client disparities: [0.009809613227844238, 0.0025564879179000854], all client disparities: [0.023188382387161255, 0.007019475102424622], all client accs: [0.7602905631065369, 0.7847766280174255],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:01,696 - utils - INFO - stage2_gradient_single_runtime: 0.006443023681640625
2023-09-28 23:28:01,699 - utils - INFO - 1, epoch: 776, all client loss: [0.5415363907814026, 0.46069464087486267], all pred client disparities: [0.010580718517303467, 0.0032136887311935425], all client disparities: [0.023188382387161255, 0.007082134485244751], all client accs: [0.7602905631065369, 0.7841545343399048],  alphas:tensor([0.4829, 0.0000, 0.0460, 0.4711], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:01,930 - utils - INFO - stage2_gradient_single_runtime: 0.0063588619232177734
2023-09-28 23:28:01,932 - utils - INFO - 1, epoch: 777, all client loss: [0.5416316986083984, 0.46078625321388245], all pred client disparities: [0.010351091623306274, 0.0031104832887649536], all client disparities: [0.023188382387161255, 0.00715520977973938], all client accs: [0.7602905631065369, 0.784123420715332],  alphas:tensor([0.4820, 0.0000, 0.0468, 0.4712], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:02,162 - utils - INFO - stage2_gradient_single_runtime: 0.006293535232543945
2023-09-28 23:28:02,168 - utils - INFO - 1, epoch: 778, all client loss: [0.541731595993042, 0.46087685227394104], all pred client disparities: [0.010111868381500244, 0.0030061304569244385], all client disparities: [0.023188382387161255, 0.00715520977973938], all client accs: [0.7602905631065369, 0.784123420715332],  alphas:tensor([0.4812, 0.0000, 0.0476, 0.4712], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:02,410 - utils - INFO - stage2_gradient_single_runtime: 0.006287097930908203
2023-09-28 23:28:02,417 - utils - INFO - 1, epoch: 779, all client loss: [0.541836142539978, 0.460966557264328], all pred client disparities: [0.009862512350082397, 0.0029006898403167725], all client disparities: [0.023188382387161255, 0.007082134485244751], all client accs: [0.7602905631065369, 0.784123420715332],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:02,546 - utils - INFO - valid: True, epoch: 779, loss: [0.5807724595069885, 0.4606174826622009], accuracy: [0.7016574740409851, 0.7846583724021912], mean_accuracy:0.7431579232215881,variance_accuracy:0.04150044918060303, disparity: [0.03030303120613098, 0.01330450177192688], mean_disparity:0.02180376648902893,variance_disparity:0.00849926471710205, pred_disparity: [0.02522742748260498, 0.000578075647354126]
2023-09-28 23:28:02,628 - utils - INFO - global_valid: True, epoch: 779,  global_loss: 0.4619533121585846, global_accuracy: 0.8140231544123043,  global_disparity:0.014927089214324951, global_pred_disparity: 0.002779930830001831,
2023-09-28 23:28:02,857 - utils - INFO - stage2_gradient_single_runtime: 0.006315708160400391
2023-09-28 23:28:02,863 - utils - INFO - 1, epoch: 780, all client loss: [0.5409846901893616, 0.46144670248031616], all pred client disparities: [0.010620743036270142, 0.0035737156867980957], all client disparities: [0.023188382387161255, 0.010245904326438904], all client accs: [0.7602905631065369, 0.7840923070907593],  alphas:tensor([0.4768, 0.0000, 0.0523, 0.4709], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:03,099 - utils - INFO - stage2_gradient_single_runtime: 0.006289005279541016
2023-09-28 23:28:03,106 - utils - INFO - 1, epoch: 781, all client loss: [0.5410855412483215, 0.46153247356414795], all pred client disparities: [0.01039460301399231, 0.0034618228673934937], all client disparities: [0.023188382387161255, 0.010245904326438904], all client accs: [0.7602905631065369, 0.7840923070907593],  alphas:tensor([0.4760, 0.0000, 0.0530, 0.4710], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:03,337 - utils - INFO - stage2_gradient_single_runtime: 0.0063364505767822266
2023-09-28 23:28:03,343 - utils - INFO - 1, epoch: 782, all client loss: [0.5411908030509949, 0.4616173803806305], all pred client disparities: [0.010158956050872803, 0.0033488869667053223], all client disparities: [0.023188382387161255, 0.008951157331466675], all client accs: [0.7602905631065369, 0.7839990258216858],  alphas:tensor([0.4752, 0.0000, 0.0538, 0.4710], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:03,624 - utils - INFO - stage2_gradient_single_runtime: 0.007128477096557617
2023-09-28 23:28:03,630 - utils - INFO - 1, epoch: 783, all client loss: [0.5413005948066711, 0.46170130372047424], all pred client disparities: [0.009913235902786255, 0.0032349377870559692], all client disparities: [0.023188382387161255, 0.009024247527122498], all client accs: [0.7602905631065369, 0.783967912197113],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:03,868 - utils - INFO - stage2_gradient_single_runtime: 0.006424665451049805
2023-09-28 23:28:03,874 - utils - INFO - 1, epoch: 784, all client loss: [0.5404669046401978, 0.4621930420398712], all pred client disparities: [0.010658204555511475, 0.003922760486602783], all client disparities: [0.023188382387161255, 0.009796947240829468], all client accs: [0.7602905631065369, 0.7831902503967285],  alphas:tensor([0.4722, 0.0000, 0.0572, 0.4705], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:04,149 - utils - INFO - stage2_gradient_single_runtime: 0.006224393844604492
2023-09-28 23:28:04,154 - utils - INFO - 1, epoch: 785, all client loss: [0.5405718684196472, 0.4622740149497986], all pred client disparities: [0.010435700416564941, 0.003803357481956482], all client disparities: [0.023188382387161255, 0.010016202926635742], all client accs: [0.7602905631065369, 0.7832213640213013],  alphas:tensor([0.4715, 0.0000, 0.0580, 0.4706], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:04,428 - utils - INFO - stage2_gradient_single_runtime: 0.006369590759277344
2023-09-28 23:28:04,433 - utils - INFO - 1, epoch: 786, all client loss: [0.540681004524231, 0.4623541533946991], all pred client disparities: [0.010203778743743896, 0.0036829859018325806], all client disparities: [0.023188382387161255, 0.010381653904914856], all client accs: [0.7602905631065369, 0.7837190628051758],  alphas:tensor([0.4707, 0.0000, 0.0587, 0.4706], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:04,683 - utils - INFO - stage2_gradient_single_runtime: 0.006270408630371094
2023-09-28 23:28:04,688 - utils - INFO - 1, epoch: 787, all client loss: [0.5407945513725281, 0.462433397769928], all pred client disparities: [0.009961843490600586, 0.0035615861415863037], all client disparities: [0.023188382387161255, 0.010172814130783081], all client accs: [0.7602905631065369, 0.7838745713233948],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:04,920 - utils - INFO - stage2_gradient_single_runtime: 0.0063474178314208984
2023-09-28 23:28:04,925 - utils - INFO - 1, epoch: 788, all client loss: [0.5399778485298157, 0.46293655037879944], all pred client disparities: [0.010693103075027466, 0.004263371229171753], all client disparities: [0.023188382387161255, 0.009504586458206177], all client accs: [0.7602905631065369, 0.7827236652374268],  alphas:tensor([0.4688, 0.0000, 0.0612, 0.4700], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:05,154 - utils - INFO - stage2_gradient_single_runtime: 0.006284236907958984
2023-09-28 23:28:05,159 - utils - INFO - 1, epoch: 789, all client loss: [0.5400856137275696, 0.4630136489868164], all pred client disparities: [0.010474294424057007, 0.004137486219406128], all client disparities: [0.023188382387161255, 0.009504586458206177], all client accs: [0.7602905631065369, 0.7827547788619995],  alphas:tensor([0.4680, 0.0000, 0.0619, 0.4701], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:05,399 - utils - INFO - stage2_gradient_single_runtime: 0.006306648254394531
2023-09-28 23:28:05,405 - utils - INFO - 1, epoch: 790, all client loss: [0.5401976108551025, 0.46308982372283936], all pred client disparities: [0.010246217250823975, 0.004010617733001709], all client disparities: [0.023188382387161255, 0.009504586458206177], all client accs: [0.7602905631065369, 0.7827547788619995],  alphas:tensor([0.4673, 0.0000, 0.0626, 0.4701], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:05,646 - utils - INFO - stage2_gradient_single_runtime: 0.006311893463134766
2023-09-28 23:28:05,651 - utils - INFO - 1, epoch: 791, all client loss: [0.5403136610984802, 0.46316516399383545], all pred client disparities: [0.010008454322814941, 0.0038827508687973022], all client disparities: [0.023188382387161255, 0.009504586458206177], all client accs: [0.7602905631065369, 0.7827858328819275],  alphas:tensor([0.4666, 0.0000, 0.0633, 0.4701], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:05,923 - utils - INFO - stage2_gradient_single_runtime: 0.006259441375732422
2023-09-28 23:28:05,928 - utils - INFO - 1, epoch: 792, all client loss: [0.5404341220855713, 0.4632396996021271], all pred client disparities: [0.009760290384292603, 0.003753975033760071], all client disparities: [0.023188382387161255, 0.009504586458206177], all client accs: [0.7602905631065369, 0.7827858328819275],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:06,174 - utils - INFO - stage2_gradient_single_runtime: 0.010240793228149414
2023-09-28 23:28:06,179 - utils - INFO - 1, epoch: 793, all client loss: [0.5396313667297363, 0.46375352144241333], all pred client disparities: [0.010493367910385132, 0.004464656114578247], all client disparities: [0.023188382387161255, 0.009191378951072693], all client accs: [0.7602905631065369, 0.7818215489387512],  alphas:tensor([0.4653, 0.0000, 0.0652, 0.4694], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:06,444 - utils - INFO - stage2_gradient_single_runtime: 0.00627446174621582
2023-09-28 23:28:06,449 - utils - INFO - 1, epoch: 794, all client loss: [0.5397453904151917, 0.463826447725296], all pred client disparities: [0.010268598794937134, 0.004332035779953003], all client disparities: [0.023188382387161255, 0.009849190711975098], all client accs: [0.7602905631065369, 0.7820082306861877],  alphas:tensor([0.4646, 0.0000, 0.0659, 0.4695], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:06,734 - utils - INFO - stage2_gradient_single_runtime: 0.006551027297973633
2023-09-28 23:28:06,740 - utils - INFO - 1, epoch: 795, all client loss: [0.5398634672164917, 0.46389853954315186], all pred client disparities: [0.010034143924713135, 0.004198417067527771], all client disparities: [0.023188382387161255, 0.009922266006469727], all client accs: [0.7602905631065369, 0.7823814749717712],  alphas:tensor([0.4639, 0.0000, 0.0666, 0.4695], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:07,001 - utils - INFO - stage2_gradient_single_runtime: 0.010587930679321289
2023-09-28 23:28:07,004 - utils - INFO - 1, epoch: 796, all client loss: [0.5399858951568604, 0.46396970748901367], all pred client disparities: [0.009789526462554932, 0.004063844680786133], all client disparities: [0.019565194845199585, 0.009567245841026306], all client accs: [0.7602905631065369, 0.7824748158454895],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:07,271 - utils - INFO - stage2_gradient_single_runtime: 0.006331205368041992
2023-09-28 23:28:07,276 - utils - INFO - 1, epoch: 797, all client loss: [0.5391986966133118, 0.46449464559555054], all pred client disparities: [0.010508984327316284, 0.004787057638168335], all client disparities: [0.019565194845199585, 0.010120674967765808], all client accs: [0.7602905631065369, 0.7816660404205322],  alphas:tensor([0.4633, 0.0000, 0.0680, 0.4687], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:07,514 - utils - INFO - stage2_gradient_single_runtime: 0.00626063346862793
2023-09-28 23:28:07,520 - utils - INFO - 1, epoch: 798, all client loss: [0.5393140316009521, 0.46456488966941833], all pred client disparities: [0.010287463665008545, 0.004649505019187927], all client disparities: [0.019565194845199585, 0.010120674967765808], all client accs: [0.7602905631065369, 0.7817282676696777],  alphas:tensor([0.4625, 0.0000, 0.0687, 0.4687], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:07,764 - utils - INFO - stage2_gradient_single_runtime: 0.006289958953857422
2023-09-28 23:28:07,769 - utils - INFO - 1, epoch: 799, all client loss: [0.5394333004951477, 0.4646342694759369], all pred client disparities: [0.010056674480438232, 0.00451090931892395], all client disparities: [0.019565194845199585, 0.008825942873954773], all client accs: [0.7602905631065369, 0.781697154045105],  alphas:tensor([0.4618, 0.0000, 0.0694, 0.4688], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:07,849 - utils - INFO - valid: True, epoch: 799, loss: [0.5808508396148682, 0.4640122652053833], accuracy: [0.7071823477745056, 0.7829813957214355], mean_accuracy:0.7450818717479706,variance_accuracy:0.037899523973464966, disparity: [0.0030303001403808594, 0.010480836033821106], mean_disparity:0.006755568087100983,variance_disparity:0.0037252679467201233, pred_disparity: [0.024347305297851562, 5.064905053586699e-05]
2023-09-28 23:28:07,991 - utils - INFO - global_valid: True, epoch: 799,  global_loss: 0.4653112292289734, global_accuracy: 0.8127560140604257,  global_disparity:0.013581350445747375, global_pred_disparity: 0.002302303910255432,
2023-09-28 23:28:08,223 - utils - INFO - stage2_gradient_single_runtime: 0.006283998489379883
2023-09-28 23:28:08,229 - utils - INFO - 1, epoch: 800, all client loss: [0.5395567417144775, 0.4647027850151062], all pred client disparities: [0.009815812110900879, 0.004371359944343567], all client disparities: [0.019565194845199585, 0.008825942873954773], all client accs: [0.7602905631065369, 0.781697154045105],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:08,462 - utils - INFO - stage2_gradient_single_runtime: 0.0063860416412353516
2023-09-28 23:28:08,468 - utils - INFO - 1, epoch: 801, all client loss: [0.5387846231460571, 0.4652387797832489], all pred client disparities: [0.010520905256271362, 0.0051065534353256226], all client disparities: [0.019565194845199585, 0.008857324719429016], all client accs: [0.7602905631065369, 0.7817593812942505],  alphas:tensor([0.4617, 0.0000, 0.0705, 0.4678], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:08,701 - utils - INFO - stage2_gradient_single_runtime: 0.006262063980102539
2023-09-28 23:28:08,705 - utils - INFO - 1, epoch: 802, all client loss: [0.5389004945755005, 0.4653068780899048], all pred client disparities: [0.010303109884262085, 0.0049648284912109375], all client disparities: [0.019565194845199585, 0.008857324719429016], all client accs: [0.7602905631065369, 0.781852662563324],  alphas:tensor([0.4610, 0.0000, 0.0712, 0.4679], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:08,940 - utils - INFO - stage2_gradient_single_runtime: 0.006514549255371094
2023-09-28 23:28:08,943 - utils - INFO - 1, epoch: 803, all client loss: [0.5390202403068542, 0.4653741121292114], all pred client disparities: [0.010075986385345459, 0.004822060465812683], all client disparities: [0.019565194845199585, 0.008930414915084839], all client accs: [0.7602905631065369, 0.7818215489387512],  alphas:tensor([0.4602, 0.0000, 0.0718, 0.4679], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:09,181 - utils - INFO - stage2_gradient_single_runtime: 0.006356954574584961
2023-09-28 23:28:09,186 - utils - INFO - 1, epoch: 804, all client loss: [0.539143979549408, 0.4654404819011688], all pred client disparities: [0.009839236736297607, 0.004678189754486084], all client disparities: [0.019565194845199585, 0.011269256472587585], all client accs: [0.7602905631065369, 0.7813860774040222],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:09,424 - utils - INFO - stage2_gradient_single_runtime: 0.006272315979003906
2023-09-28 23:28:09,430 - utils - INFO - 1, epoch: 805, all client loss: [0.5383866429328918, 0.4659874737262726], all pred client disparities: [0.01052945852279663, 0.005424961447715759], all client disparities: [0.019565194845199585, 0.006372302770614624], all client accs: [0.7602905631065369, 0.7819148898124695],  alphas:tensor([0.4605, 0.0000, 0.0726, 0.4668], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:09,671 - utils - INFO - stage2_gradient_single_runtime: 0.006313323974609375
2023-09-28 23:28:09,676 - utils - INFO - 1, epoch: 806, all client loss: [0.5385023951530457, 0.46605390310287476], all pred client disparities: [0.010315388441085815, 0.0052797794342041016], all client disparities: [0.019565194845199585, 0.006372302770614624], all client accs: [0.7602905631065369, 0.7819460034370422],  alphas:tensor([0.4598, 0.0000, 0.0733, 0.4669], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:09,950 - utils - INFO - stage2_gradient_single_runtime: 0.010552167892456055
2023-09-28 23:28:09,955 - utils - INFO - 1, epoch: 807, all client loss: [0.5386219024658203, 0.4661194384098053], all pred client disparities: [0.010092407464981079, 0.005133450031280518], all client disparities: [0.019565194845199585, 0.00871114432811737], all client accs: [0.7602905631065369, 0.7817904949188232],  alphas:tensor([0.4591, 0.0000, 0.0740, 0.4670], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:10,207 - utils - INFO - stage2_gradient_single_runtime: 0.006255388259887695
2023-09-28 23:28:10,212 - utils - INFO - 1, epoch: 808, all client loss: [0.5387453436851501, 0.4661840796470642], all pred client disparities: [0.009859919548034668, 0.00498606264591217], all client disparities: [0.019565194845199585, 0.00871114432811737], all client accs: [0.7602905631065369, 0.7817904949188232],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:10,492 - utils - INFO - stage2_gradient_single_runtime: 0.006314516067504883
2023-09-28 23:28:10,497 - utils - INFO - 1, epoch: 809, all client loss: [0.5380024909973145, 0.4667421877384186], all pred client disparities: [0.01053464412689209, 0.00574396550655365], all client disparities: [0.019565194845199585, 0.005422145128250122], all client accs: [0.7602905631065369, 0.7818837761878967],  alphas:tensor([0.4597, 0.0000, 0.0745, 0.4658], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:10,767 - utils - INFO - stage2_gradient_single_runtime: 0.006308794021606445
2023-09-28 23:28:10,773 - utils - INFO - 1, epoch: 810, all client loss: [0.538117527961731, 0.46680739521980286], all pred client disparities: [0.010324656963348389, 0.00559602677822113], all client disparities: [0.019565194845199585, 0.005495235323905945], all client accs: [0.7602905631065369, 0.781852662563324],  alphas:tensor([0.4589, 0.0000, 0.0752, 0.4658], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:11,005 - utils - INFO - stage2_gradient_single_runtime: 0.006296873092651367
2023-09-28 23:28:11,009 - utils - INFO - 1, epoch: 811, all client loss: [0.5382362008094788, 0.4668716490268707], all pred client disparities: [0.010105997323989868, 0.005446895956993103], all client disparities: [0.019565194845199585, 0.005933776497840881], all client accs: [0.7602905631065369, 0.7817593812942505],  alphas:tensor([0.4582, 0.0000, 0.0759, 0.4659], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:11,238 - utils - INFO - stage2_gradient_single_runtime: 0.0062580108642578125
2023-09-28 23:28:11,241 - utils - INFO - 1, epoch: 812, all client loss: [0.538358747959137, 0.46693503856658936], all pred client disparities: [0.009878098964691162, 0.005296558141708374], all client disparities: [0.019565194845199585, 0.006006866693496704], all client accs: [0.7602905631065369, 0.7817593812942505],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:11,472 - utils - INFO - stage2_gradient_single_runtime: 0.006929159164428711
2023-09-28 23:28:11,476 - utils - INFO - 1, epoch: 813, all client loss: [0.537630021572113, 0.46750426292419434], all pred client disparities: [0.010536789894104004, 0.0060653239488601685], all client disparities: [0.019565194845199585, 0.003657594323158264], all client accs: [0.7602905631065369, 0.7784932255744934],  alphas:tensor([0.4591, 0.0000, 0.0763, 0.4646], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:11,708 - utils - INFO - stage2_gradient_single_runtime: 0.0067272186279296875
2023-09-28 23:28:11,711 - utils - INFO - 1, epoch: 814, all client loss: [0.537743866443634, 0.46756863594055176], all pred client disparities: [0.01033097505569458, 0.005915268789976835], all client disparities: [0.019565194845199585, 0.003730684518814087], all client accs: [0.7602905631065369, 0.7784621119499207],  alphas:tensor([0.4583, 0.0000, 0.0770, 0.4647], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:11,955 - utils - INFO - stage2_gradient_single_runtime: 0.0064318180084228516
2023-09-28 23:28:11,958 - utils - INFO - 1, epoch: 815, all client loss: [0.5378612279891968, 0.467631995677948], all pred client disparities: [0.01011696457862854, 0.0057639628648757935], all client disparities: [0.019565194845199585, 0.005129799246788025], all client accs: [0.7602905631065369, 0.7814483046531677],  alphas:tensor([0.4576, 0.0000, 0.0776, 0.4648], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:12,197 - utils - INFO - stage2_gradient_single_runtime: 0.00626063346862793
2023-09-28 23:28:12,201 - utils - INFO - 1, epoch: 816, all client loss: [0.5379822850227356, 0.4676945209503174], all pred client disparities: [0.009894073009490967, 0.005611389875411987], all client disparities: [0.019565194845199585, 0.005129799246788025], all client accs: [0.7602905631065369, 0.7814483046531677],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:12,429 - utils - INFO - stage2_gradient_single_runtime: 0.006262779235839844
2023-09-28 23:28:12,433 - utils - INFO - 1, epoch: 817, all client loss: [0.5372675657272339, 0.4682749807834625], all pred client disparities: [0.01053592562675476, 0.006390690803527832], all client disparities: [0.019565194845199585, 0.004440724849700928], all client accs: [0.7602905631065369, 0.7781510353088379],  alphas:tensor([0.4587, 0.0000, 0.0779, 0.4634], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:12,692 - utils - INFO - stage2_gradient_single_runtime: 0.010371208190917969
2023-09-28 23:28:12,696 - utils - INFO - 1, epoch: 818, all client loss: [0.5373796224594116, 0.46833884716033936], all pred client disparities: [0.010334759950637817, 0.006239205598831177], all client disparities: [0.019565194845199585, 0.004440724849700928], all client accs: [0.7602905631065369, 0.7781821489334106],  alphas:tensor([0.4580, 0.0000, 0.0786, 0.4634], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:12,930 - utils - INFO - stage2_gradient_single_runtime: 0.006242513656616211
2023-09-28 23:28:12,933 - utils - INFO - 1, epoch: 819, all client loss: [0.5374951362609863, 0.46840178966522217], all pred client disparities: [0.01012563705444336, 0.006086379289627075], all client disparities: [0.019565194845199585, 0.004440724849700928], all client accs: [0.7602905631065369, 0.7781510353088379],  alphas:tensor([0.4572, 0.0000, 0.0793, 0.4635], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:13,013 - utils - INFO - valid: True, epoch: 819, loss: [0.5807948112487793, 0.4679296612739563], accuracy: [0.7071823477745056, 0.779254674911499], mean_accuracy:0.7432185113430023,variance_accuracy:0.036036163568496704, disparity: [0.0030303001403808594, 0.004833504557609558], mean_disparity:0.003931902348995209,variance_disparity:0.0009016022086143494, pred_disparity: [0.022710144519805908, 0.001275286078453064]
2023-09-28 23:28:13,140 - utils - INFO - global_valid: True, epoch: 819,  global_loss: 0.46918439865112305, global_accuracy: 0.8118828307615618,  global_disparity:0.00808088481426239, global_pred_disparity: 0.0010662823915481567,
2023-09-28 23:28:13,368 - utils - INFO - stage2_gradient_single_runtime: 0.0061893463134765625
2023-09-28 23:28:13,371 - utils - INFO - 1, epoch: 820, all client loss: [0.5376142263412476, 0.4684637784957886], all pred client disparities: [0.009908020496368408, 0.005932226311415434], all client disparities: [0.019565194845199585, 0.0045138150453567505], all client accs: [0.7602905631065369, 0.7781199216842651],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:13,600 - utils - INFO - stage2_gradient_single_runtime: 0.006282329559326172
2023-09-28 23:28:13,604 - utils - INFO - 1, epoch: 821, all client loss: [0.5369134545326233, 0.4690555930137634], all pred client disparities: [0.010532468557357788, 0.006721809506416321], all client disparities: [0.019565194845199585, 0.004137933254241943], all client accs: [0.7602905631065369, 0.7764090895652771],  alphas:tensor([0.4585, 0.0000, 0.0795, 0.4620], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:13,791 - utils - INFO - stage2_gradient_single_runtime: 0.006296634674072266
2023-09-28 23:28:13,795 - utils - INFO - 1, epoch: 822, all client loss: [0.5370232462882996, 0.4691193103790283], all pred client disparities: [0.010336250066757202, 0.006569504737854004], all client disparities: [0.019565194845199585, 0.0047226399183273315], all client accs: [0.7602905631065369, 0.7776222229003906],  alphas:tensor([0.4578, 0.0000, 0.0801, 0.4621], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:14,043 - utils - INFO - stage2_gradient_single_runtime: 0.006284236907958984
2023-09-28 23:28:14,049 - utils - INFO - 1, epoch: 823, all client loss: [0.537136435508728, 0.4691821336746216], all pred client disparities: [0.010132312774658203, 0.006415769457817078], all client disparities: [0.019565194845199585, 0.004148364067077637], all client accs: [0.7602905631065369, 0.7778089046478271],  alphas:tensor([0.4570, 0.0000, 0.0808, 0.4622], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:14,298 - utils - INFO - stage2_gradient_single_runtime: 0.006400108337402344
2023-09-28 23:28:14,303 - utils - INFO - 1, epoch: 824, all client loss: [0.5372530221939087, 0.46924400329589844], all pred client disparities: [0.009920299053192139, 0.0062606483697891235], all client disparities: [0.019565194845199585, 0.004148364067077637], all client accs: [0.7602905631065369, 0.7778089046478271],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:14,542 - utils - INFO - stage2_gradient_single_runtime: 0.0064983367919921875
2023-09-28 23:28:14,548 - utils - INFO - 1, epoch: 825, all client loss: [0.5365660190582275, 0.46984729170799255], all pred client disparities: [0.010526537895202637, 0.007060334086418152], all client disparities: [0.019565194845199585, 0.00417974591255188], all client accs: [0.7602905631065369, 0.7760358452796936],  alphas:tensor([0.4585, 0.0000, 0.0810, 0.4605], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:14,793 - utils - INFO - stage2_gradient_single_runtime: 0.0063018798828125
2023-09-28 23:28:14,799 - utils - INFO - 1, epoch: 826, all client loss: [0.5366733074188232, 0.46991127729415894], all pred client disparities: [0.010335534811019897, 0.006907850503921509], all client disparities: [0.019565194845199585, 0.00417974591255188], all client accs: [0.7602905631065369, 0.7763468623161316],  alphas:tensor([0.4578, 0.0000, 0.0816, 0.4606], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:15,038 - utils - INFO - stage2_gradient_single_runtime: 0.006306648254394531
2023-09-28 23:28:15,044 - utils - INFO - 1, epoch: 827, all client loss: [0.5367836356163025, 0.46997424960136414], all pred client disparities: [0.01013725996017456, 0.006753891706466675], all client disparities: [0.019565194845199585, 0.004399016499519348], all client accs: [0.7602905631065369, 0.7762846946716309],  alphas:tensor([0.4570, 0.0000, 0.0822, 0.4607], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:15,277 - utils - INFO - stage2_gradient_single_runtime: 0.006272077560424805
2023-09-28 23:28:15,280 - utils - INFO - 1, epoch: 828, all client loss: [0.5368974208831787, 0.4700363576412201], all pred client disparities: [0.009931296110153198, 0.006598427891731262], all client disparities: [0.019565194845199585, 0.003678560256958008], all client accs: [0.7602905631065369, 0.7762535810470581],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:15,520 - utils - INFO - stage2_gradient_single_runtime: 0.00628662109375
2023-09-28 23:28:15,523 - utils - INFO - 1, epoch: 829, all client loss: [0.5362240672111511, 0.4706512987613678], all pred client disparities: [0.010518640279769897, 0.007407978177070618], all client disparities: [0.019565194845199585, 0.0029372423887252808], all client accs: [0.7602905631065369, 0.7758802771568298],  alphas:tensor([0.4586, 0.0000, 0.0824, 0.4590], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:15,760 - utils - INFO - stage2_gradient_single_runtime: 0.006479501724243164
2023-09-28 23:28:15,765 - utils - INFO - 1, epoch: 830, all client loss: [0.5363282561302185, 0.47071582078933716], all pred client disparities: [0.010333150625228882, 0.007255971431732178], all client disparities: [0.019565194845199585, 0.0029372423887252808], all client accs: [0.7602905631065369, 0.7758802771568298],  alphas:tensor([0.4579, 0.0000, 0.0830, 0.4591], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:15,997 - utils - INFO - stage2_gradient_single_runtime: 0.007335186004638672
2023-09-28 23:28:16,000 - utils - INFO - 1, epoch: 831, all client loss: [0.5364354848861694, 0.4707793593406677], all pred client disparities: [0.010140746831893921, 0.007102400064468384], all client disparities: [0.019565194845199585, 0.003887385129928589], all client accs: [0.7602905631065369, 0.7756625413894653],  alphas:tensor([0.4572, 0.0000, 0.0836, 0.4592], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:16,251 - utils - INFO - stage2_gradient_single_runtime: 0.0063855648040771484
2023-09-28 23:28:16,256 - utils - INFO - 1, epoch: 832, all client loss: [0.5365458726882935, 0.47084200382232666], all pred client disparities: [0.0099412202835083, 0.00694727897644043], all client disparities: [0.019565194845199585, 0.003887385129928589], all client accs: [0.7602905631065369, 0.7756625413894653],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:16,496 - utils - INFO - stage2_gradient_single_runtime: 0.006247758865356445
2023-09-28 23:28:16,501 - utils - INFO - 1, epoch: 833, all client loss: [0.5358863472938538, 0.47146880626678467], all pred client disparities: [0.010508984327316284, 0.007766500115394592], all client disparities: [0.019565194845199585, 0.012870565056800842], all client accs: [0.7602905631065369, 0.7849633097648621],  alphas:tensor([0.4588, 0.0000, 0.0839, 0.4573], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:16,729 - utils - INFO - stage2_gradient_single_runtime: 0.006340980529785156
2023-09-28 23:28:16,732 - utils - INFO - 1, epoch: 834, all client loss: [0.5359870791435242, 0.47153419256210327], all pred client disparities: [0.010329335927963257, 0.007615596055984497], all client disparities: [0.019565194845199585, 0.012870565056800842], all client accs: [0.7602905631065369, 0.7849944233894348],  alphas:tensor([0.4582, 0.0000, 0.0844, 0.4574], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:16,964 - utils - INFO - stage2_gradient_single_runtime: 0.0062787532806396484
2023-09-28 23:28:16,969 - utils - INFO - 1, epoch: 835, all client loss: [0.5360907912254333, 0.4715985655784607], all pred client disparities: [0.010143309831619263, 0.007463023066520691], all client disparities: [0.019565194845199585, 0.002571791410446167], all client accs: [0.7602905631065369, 0.7756625413894653],  alphas:tensor([0.4575, 0.0000, 0.0850, 0.4575], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:17,198 - utils - INFO - stage2_gradient_single_runtime: 0.006209850311279297
2023-09-28 23:28:17,201 - utils - INFO - 1, epoch: 836, all client loss: [0.5361974835395813, 0.4716620147228241], all pred client disparities: [0.009950459003448486, 0.0073089152574539185], all client disparities: [0.019565194845199585, 0.002571791410446167], all client accs: [0.7602905631065369, 0.7757247686386108],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:17,439 - utils - INFO - stage2_gradient_single_runtime: 0.006262063980102539
2023-09-28 23:28:17,444 - utils - INFO - 1, epoch: 837, all client loss: [0.5355516076087952, 0.4723009467124939], all pred client disparities: [0.010497957468032837, 0.008137628436088562], all client disparities: [0.019565194845199585, 0.013987749814987183], all client accs: [0.7602905631065369, 0.7851499319076538],  alphas:tensor([0.4592, 0.0000, 0.0853, 0.4555], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:17,683 - utils - INFO - stage2_gradient_single_runtime: 0.006318330764770508
2023-09-28 23:28:17,687 - utils - INFO - 1, epoch: 838, all client loss: [0.5356486439704895, 0.47236746549606323], all pred client disparities: [0.010324478149414062, 0.007988423109054565], all client disparities: [0.019565194845199585, 0.013987749814987183], all client accs: [0.7602905631065369, 0.7852121591567993],  alphas:tensor([0.4586, 0.0000, 0.0858, 0.4556], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:17,916 - utils - INFO - stage2_gradient_single_runtime: 0.0073468685150146484
2023-09-28 23:28:17,919 - utils - INFO - 1, epoch: 839, all client loss: [0.5357484221458435, 0.4724330008029938], all pred client disparities: [0.010145038366317749, 0.007837548851966858], all client disparities: [0.019565194845199585, 0.013987749814987183], all client accs: [0.7602905631065369, 0.7852743268013],  alphas:tensor([0.4579, 0.0000, 0.0863, 0.4557], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:18,048 - utils - INFO - valid: True, epoch: 839, loss: [0.5809460282325745, 0.4721227288246155], accuracy: [0.7127072215080261, 0.7857142686843872], mean_accuracy:0.7492107450962067,variance_accuracy:0.03650352358818054, disparity: [0.0015151500701904297, 0.006565272808074951], mean_disparity:0.00404021143913269,variance_disparity:0.0025250613689422607, pred_disparity: [0.021251827478408813, 0.0028121471405029297]
2023-09-28 23:28:18,137 - utils - INFO - global_valid: True, epoch: 839,  global_loss: 0.47333255410194397, global_accuracy: 0.8108810536579083,  global_disparity:0.0032886862754821777, global_pred_disparity: 0.00038510560989379883,
2023-09-28 23:28:18,371 - utils - INFO - stage2_gradient_single_runtime: 0.006279945373535156
2023-09-28 23:28:18,376 - utils - INFO - 1, epoch: 840, all client loss: [0.5358510613441467, 0.4724975526332855], all pred client disparities: [0.009959280490875244, 0.0076850056648254395], all client disparities: [0.019565194845199585, 0.013309091329574585], all client accs: [0.7602905631065369, 0.7845900058746338],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:18,615 - utils - INFO - stage2_gradient_single_runtime: 0.006472349166870117
2023-09-28 23:28:18,622 - utils - INFO - 1, epoch: 841, all client loss: [0.5352190732955933, 0.4731489419937134], all pred client disparities: [0.010485947132110596, 0.00852307677268982], all client disparities: [0.01775360107421875, 0.014133930206298828], all client accs: [0.7627118825912476, 0.7846211194992065],  alphas:tensor([0.4597, 0.0000, 0.0867, 0.4536], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:18,862 - utils - INFO - stage2_gradient_single_runtime: 0.006467342376708984
2023-09-28 23:28:18,869 - utils - INFO - 1, epoch: 842, all client loss: [0.5353120565414429, 0.47321683168411255], all pred client disparities: [0.010318905115127563, 0.00837622582912445], all client disparities: [0.01775360107421875, 0.014133930206298828], all client accs: [0.7627118825912476, 0.7845900058746338],  alphas:tensor([0.4591, 0.0000, 0.0872, 0.4537], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:19,104 - utils - INFO - stage2_gradient_single_runtime: 0.0062944889068603516
2023-09-28 23:28:19,111 - utils - INFO - 1, epoch: 843, all client loss: [0.5354076027870178, 0.4732838273048401], all pred client disparities: [0.01014643907546997, 0.008227646350860596], all client disparities: [0.01775360107421875, 0.014133930206298828], all client accs: [0.7627118825912476, 0.7847455739974976],  alphas:tensor([0.4585, 0.0000, 0.0877, 0.4539], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:19,346 - utils - INFO - stage2_gradient_single_runtime: 0.00637364387512207
2023-09-28 23:28:19,352 - utils - INFO - 1, epoch: 844, all client loss: [0.5355057716369629, 0.47334980964660645], all pred client disparities: [0.009968101978302002, 0.008077368140220642], all client disparities: [0.01775360107421875, 0.014133930206298828], all client accs: [0.7627118825912476, 0.7847455739974976],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:19,589 - utils - INFO - stage2_gradient_single_runtime: 0.006279706954956055
2023-09-28 23:28:19,596 - utils - INFO - 1, epoch: 845, all client loss: [0.5348877310752869, 0.4740138649940491], all pred client disparities: [0.010473400354385376, 0.00892457365989685], all client disparities: [0.01775360107421875, 0.011116325855255127], all client accs: [0.7627118825912476, 0.783687949180603],  alphas:tensor([0.4603, 0.0000, 0.0881, 0.4516], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:19,832 - utils - INFO - stage2_gradient_single_runtime: 0.006299495697021484
2023-09-28 23:28:19,838 - utils - INFO - 1, epoch: 846, all client loss: [0.5349763035774231, 0.47408348321914673], all pred client disparities: [0.010313183069229126, 0.008780673146247864], all client disparities: [0.01775360107421875, 0.013204619288444519], all client accs: [0.7627118825912476, 0.7835013270378113],  alphas:tensor([0.4597, 0.0000, 0.0885, 0.4517], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:20,076 - utils - INFO - stage2_gradient_single_runtime: 0.006279945373535156
2023-09-28 23:28:20,083 - utils - INFO - 1, epoch: 847, all client loss: [0.5350673198699951, 0.474152147769928], all pred client disparities: [0.010147809982299805, 0.008635088801383972], all client disparities: [0.01775360107421875, 0.013204619288444519], all client accs: [0.7627118825912476, 0.7835013270378113],  alphas:tensor([0.4591, 0.0000, 0.0890, 0.4519], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:20,317 - utils - INFO - stage2_gradient_single_runtime: 0.006259918212890625
2023-09-28 23:28:20,324 - utils - INFO - 1, epoch: 848, all client loss: [0.5351608395576477, 0.4742198586463928], all pred client disparities: [0.009977161884307861, 0.008487701416015625], all client disparities: [0.01775360107421875, 0.014280110597610474], all client accs: [0.7627118825912476, 0.7843100428581238],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:20,556 - utils - INFO - stage2_gradient_single_runtime: 0.006406307220458984
2023-09-28 23:28:20,560 - utils - INFO - 1, epoch: 849, all client loss: [0.5345568656921387, 0.4748968780040741], all pred client disparities: [0.01046067476272583, 0.009343758225440979], all client disparities: [0.015942007303237915, 0.011346027255058289], all client accs: [0.7651332020759583, 0.7829102873802185],  alphas:tensor([0.7364, 0.0000, 0.0000, 0.2636], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:20,798 - utils - INFO - stage2_gradient_single_runtime: 0.006685972213745117
2023-09-28 23:28:20,805 - utils - INFO - 1, epoch: 850, all client loss: [0.5346521139144897, 0.47457805275917053], all pred client disparities: [0.010412544012069702, 0.009093642234802246], all client disparities: [0.01775360107421875, 0.011993393301963806], all client accs: [0.7627118825912476, 0.7830657958984375],  alphas:tensor([0.4606, 0.0000, 0.0893, 0.4502], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:21,039 - utils - INFO - stage2_gradient_single_runtime: 0.006285667419433594
2023-09-28 23:28:21,047 - utils - INFO - 1, epoch: 851, all client loss: [0.534737765789032, 0.47464892268180847], all pred client disparities: [0.010256558656692505, 0.008952289819717407], all client disparities: [0.01775360107421875, 0.011993393301963806], all client accs: [0.7627118825912476, 0.7830657958984375],  alphas:tensor([0.4600, 0.0000, 0.0897, 0.4503], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:21,300 - utils - INFO - stage2_gradient_single_runtime: 0.007146358489990234
2023-09-28 23:28:21,303 - utils - INFO - 1, epoch: 852, all client loss: [0.5348257422447205, 0.47471883893013], all pred client disparities: [0.010095834732055664, 0.008809179067611694], all client disparities: [0.015942007303237915, 0.014154776930809021], all client accs: [0.7651332020759583, 0.7829102873802185],  alphas:tensor([0.4594, 0.0000, 0.0902, 0.4504], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:21,551 - utils - INFO - stage2_gradient_single_runtime: 0.007344722747802734
2023-09-28 23:28:21,554 - utils - INFO - 1, epoch: 853, all client loss: [0.5349161028862, 0.4747878611087799], all pred client disparities: [0.009930163621902466, 0.00866428017616272], all client disparities: [0.015942007303237915, 0.014154776930809021], all client accs: [0.7651332020759583, 0.7828791737556458],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:21,789 - utils - INFO - stage2_gradient_single_runtime: 0.00628972053527832
2023-09-28 23:28:21,794 - utils - INFO - 1, epoch: 854, all client loss: [0.5343226790428162, 0.4754723310470581], all pred client disparities: [0.010398060083389282, 0.009524226188659668], all client disparities: [0.015942007303237915, 0.01149220671504736], all client accs: [0.7651332020759583, 0.782692551612854],  alphas:tensor([0.7473, 0.0000, 0.0000, 0.2527], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:22,029 - utils - INFO - stage2_gradient_single_runtime: 0.006368398666381836
2023-09-28 23:28:22,034 - utils - INFO - 1, epoch: 855, all client loss: [0.5344094634056091, 0.47514769434928894], all pred client disparities: [0.010360956192016602, 0.00928013026714325], all client disparities: [0.015942007303237915, 0.011346027255058289], all client accs: [0.7651332020759583, 0.7828791737556458],  alphas:tensor([0.7412, 0.0000, 0.0000, 0.2588], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:22,271 - utils - INFO - stage2_gradient_single_runtime: 0.006258964538574219
2023-09-28 23:28:22,276 - utils - INFO - 1, epoch: 856, all client loss: [0.5345004796981812, 0.4748266637325287], all pred client disparities: [0.010318458080291748, 0.009033232927322388], all client disparities: [0.015942007303237915, 0.011993393301963806], all client accs: [0.7651332020759583, 0.7829102873802185],  alphas:tensor([0.7352, 0.0000, 0.0000, 0.2648], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:22,511 - utils - INFO - stage2_gradient_single_runtime: 0.006285905838012695
2023-09-28 23:28:22,516 - utils - INFO - 1, epoch: 857, all client loss: [0.5345959663391113, 0.4745091497898102], all pred client disparities: [0.010270476341247559, 0.008783623576164246], all client disparities: [0.015942007303237915, 0.011993393301963806], all client accs: [0.7651332020759583, 0.7830657958984375],  alphas:tensor([0.4600, 0.0000, 0.0901, 0.4499], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:22,752 - utils - INFO - stage2_gradient_single_runtime: 0.006340503692626953
2023-09-28 23:28:22,757 - utils - INFO - 1, epoch: 858, all client loss: [0.534681499004364, 0.4745801091194153], all pred client disparities: [0.010113924741744995, 0.008643046021461487], all client disparities: [0.015942007303237915, 0.014154776930809021], all client accs: [0.7651332020759583, 0.7829414010047913],  alphas:tensor([0.4594, 0.0000, 0.0905, 0.4500], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:22,994 - utils - INFO - stage2_gradient_single_runtime: 0.00630950927734375
2023-09-28 23:28:22,999 - utils - INFO - 1, epoch: 859, all client loss: [0.5347693562507629, 0.47465014457702637], all pred client disparities: [0.009952664375305176, 0.008500829339027405], all client disparities: [0.015942007303237915, 0.014154776930809021], all client accs: [0.7651332020759583, 0.7828791737556458],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:23,080 - utils - INFO - valid: True, epoch: 859, loss: [0.5809536576271057, 0.47506994009017944], accuracy: [0.7292817831039429, 0.7852795124053955], mean_accuracy:0.7572806477546692,variance_accuracy:0.02799886465072632, disparity: [0.015151500701904297, 0.008958160877227783], mean_disparity:0.01205483078956604,variance_disparity:0.003096669912338257, pred_disparity: [0.019559741020202637, 0.004420757293701172]
2023-09-28 23:28:23,208 - utils - INFO - global_valid: True, epoch: 859,  global_loss: 0.47624707221984863, global_accuracy: 0.8104591823399515,  global_disparity:0.006067097187042236, global_pred_disparity: 0.0018992871046066284,
2023-09-28 23:28:23,442 - utils - INFO - stage2_gradient_single_runtime: 0.006245613098144531
2023-09-28 23:28:23,445 - utils - INFO - 1, epoch: 860, all client loss: [0.5341814756393433, 0.4753309190273285], all pred client disparities: [0.01040831208229065, 0.009356409311294556], all client disparities: [0.015942007303237915, 0.011419117450714111], all client accs: [0.7651332020759583, 0.782692551612854],  alphas:tensor([0.7482, 0.0000, 0.0000, 0.2518], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:23,681 - utils - INFO - stage2_gradient_single_runtime: 0.007162570953369141
2023-09-28 23:28:23,684 - utils - INFO - 1, epoch: 861, all client loss: [0.5342660546302795, 0.4750079810619354], all pred client disparities: [0.010374993085861206, 0.009114786982536316], all client disparities: [0.015942007303237915, 0.011346027255058289], all client accs: [0.7651332020759583, 0.782972514629364],  alphas:tensor([0.7421, 0.0000, 0.0000, 0.2579], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:23,929 - utils - INFO - stage2_gradient_single_runtime: 0.006692171096801758
2023-09-28 23:28:23,935 - utils - INFO - 1, epoch: 862, all client loss: [0.5343548059463501, 0.47468864917755127], all pred client disparities: [0.010336548089981079, 0.00887046754360199], all client disparities: [0.015942007303237915, 0.011993393301963806], all client accs: [0.7651332020759583, 0.7829414010047913],  alphas:tensor([0.7362, 0.0000, 0.0000, 0.2638], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:24,199 - utils - INFO - stage2_gradient_single_runtime: 0.006314754486083984
2023-09-28 23:28:24,204 - utils - INFO - 1, epoch: 863, all client loss: [0.5344479084014893, 0.47437286376953125], all pred client disparities: [0.010292738676071167, 0.008623465895652771], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.783407986164093],  alphas:tensor([0.4600, 0.0000, 0.0905, 0.4495], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:24,444 - utils - INFO - stage2_gradient_single_runtime: 0.006345510482788086
2023-09-28 23:28:24,450 - utils - INFO - 1, epoch: 864, all client loss: [0.5345308184623718, 0.4744448661804199], all pred client disparities: [0.010140687227249146, 0.008485585451126099], all client disparities: [0.015942007303237915, 0.013570070266723633], all client accs: [0.7651332020759583, 0.7833146452903748],  alphas:tensor([0.4594, 0.0000, 0.0909, 0.4496], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:24,691 - utils - INFO - stage2_gradient_single_runtime: 0.006303071975708008
2023-09-28 23:28:24,696 - utils - INFO - 1, epoch: 865, all client loss: [0.5346159934997559, 0.47451603412628174], all pred client disparities: [0.009984105825424194, 0.008346125483512878], all client disparities: [0.015942007303237915, 0.013570070266723633], all client accs: [0.7651332020759583, 0.783283531665802],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:24,933 - utils - INFO - stage2_gradient_single_runtime: 0.006298542022705078
2023-09-28 23:28:24,938 - utils - INFO - 1, epoch: 866, all client loss: [0.5340338349342346, 0.47519323229789734], all pred client disparities: [0.010426968336105347, 0.009197533130645752], all client disparities: [0.015942007303237915, 0.011565297842025757], all client accs: [0.7651332020759583, 0.7826303243637085],  alphas:tensor([0.7493, 0.0000, 0.0000, 0.2507], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:25,177 - utils - INFO - stage2_gradient_single_runtime: 0.006288290023803711
2023-09-28 23:28:25,183 - utils - INFO - 1, epoch: 867, all client loss: [0.5341160297393799, 0.4748719036579132], all pred client disparities: [0.010397791862487793, 0.008958563208580017], all client disparities: [0.015942007303237915, 0.011993393301963806], all client accs: [0.7651332020759583, 0.7829102873802185],  alphas:tensor([0.7433, 0.0000, 0.0000, 0.2567], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:25,421 - utils - INFO - stage2_gradient_single_runtime: 0.006268739700317383
2023-09-28 23:28:25,426 - utils - INFO - 1, epoch: 868, all client loss: [0.5342022776603699, 0.4745542109012604], all pred client disparities: [0.010363548994064331, 0.008716970682144165], all client disparities: [0.015942007303237915, 0.011993393301963806], all client accs: [0.7651332020759583, 0.7829414010047913],  alphas:tensor([0.7374, 0.0000, 0.0000, 0.2626], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:25,664 - utils - INFO - stage2_gradient_single_runtime: 0.006302595138549805
2023-09-28 23:28:25,669 - utils - INFO - 1, epoch: 869, all client loss: [0.5342927575111389, 0.4742400646209717], all pred client disparities: [0.010324209928512573, 0.008472815155982971], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.7834390997886658],  alphas:tensor([0.4601, 0.0000, 0.0908, 0.4491], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:25,904 - utils - INFO - stage2_gradient_single_runtime: 0.006396293640136719
2023-09-28 23:28:25,910 - utils - INFO - 1, epoch: 870, all client loss: [0.5343729257583618, 0.47431325912475586], all pred client disparities: [0.010176926851272583, 0.008337780833244324], all client disparities: [0.015942007303237915, 0.013570070266723633], all client accs: [0.7651332020759583, 0.7833457589149475],  alphas:tensor([0.4595, 0.0000, 0.0913, 0.4492], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:26,148 - utils - INFO - stage2_gradient_single_runtime: 0.0062906742095947266
2023-09-28 23:28:26,153 - utils - INFO - 1, epoch: 871, all client loss: [0.5344551801681519, 0.4743855893611908], all pred client disparities: [0.010025441646575928, 0.008201196789741516], all client disparities: [0.015942007303237915, 0.013350799679756165], all client accs: [0.7651332020759583, 0.7834390997886658],  alphas:tensor([0.4590, 0.0000, 0.0917, 0.4493], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:26,393 - utils - INFO - stage2_gradient_single_runtime: 0.006287813186645508
2023-09-28 23:28:26,398 - utils - INFO - 1, epoch: 872, all client loss: [0.5345396399497986, 0.4744570553302765], all pred client disparities: [0.009869575500488281, 0.008063122630119324], all client disparities: [0.015942007303237915, 0.013350799679756165], all client accs: [0.7651332020759583, 0.7834390997886658],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:26,588 - utils - INFO - stage2_gradient_single_runtime: 0.006239175796508789
2023-09-28 23:28:26,593 - utils - INFO - 1, epoch: 873, all client loss: [0.5339619517326355, 0.47513052821159363], all pred client disparities: [0.010306745767593384, 0.008909061551094055], all client disparities: [0.015942007303237915, 0.011346027255058289], all client accs: [0.7651332020759583, 0.7828791737556458],  alphas:tensor([0.7487, 0.0000, 0.0000, 0.2513], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:26,826 - utils - INFO - stage2_gradient_single_runtime: 0.006284475326538086
2023-09-28 23:28:26,831 - utils - INFO - 1, epoch: 874, all client loss: [0.5340437293052673, 0.4748103618621826], all pred client disparities: [0.010278373956680298, 0.008670926094055176], all client disparities: [0.015942007303237915, 0.011993393301963806], all client accs: [0.7651332020759583, 0.7829414010047913],  alphas:tensor([0.7427, 0.0000, 0.0000, 0.2573], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:27,061 - utils - INFO - stage2_gradient_single_runtime: 0.006327390670776367
2023-09-28 23:28:27,067 - utils - INFO - 1, epoch: 875, all client loss: [0.5341296792030334, 0.4744938313961029], all pred client disparities: [0.010245174169540405, 0.008430302143096924], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.7833457589149475],  alphas:tensor([0.7367, 0.0000, 0.0000, 0.2633], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:27,299 - utils - INFO - stage2_gradient_single_runtime: 0.006295442581176758
2023-09-28 23:28:27,304 - utils - INFO - 1, epoch: 876, all client loss: [0.5342196822166443, 0.4741809368133545], all pred client disparities: [0.010206818580627441, 0.00818723440170288], all client disparities: [0.015942007303237915, 0.013570070266723633], all client accs: [0.7651332020759583, 0.7833457589149475],  alphas:tensor([0.4596, 0.0000, 0.0916, 0.4488], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:27,534 - utils - INFO - stage2_gradient_single_runtime: 0.00638580322265625
2023-09-28 23:28:27,539 - utils - INFO - 1, epoch: 877, all client loss: [0.5342992544174194, 0.47425439953804016], all pred client disparities: [0.010060012340545654, 0.008053436875343323], all client disparities: [0.015942007303237915, 0.013350799679756165], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.4591, 0.0000, 0.0920, 0.4489], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:27,772 - utils - INFO - stage2_gradient_single_runtime: 0.00629734992980957
2023-09-28 23:28:27,777 - utils - INFO - 1, epoch: 878, all client loss: [0.5343808531761169, 0.47432708740234375], all pred client disparities: [0.009909063577651978, 0.007918208837509155], all client disparities: [0.015942007303237915, 0.013350799679756165], all client accs: [0.7651332020759583, 0.7834390997886658],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:28,012 - utils - INFO - stage2_gradient_single_runtime: 0.006277799606323242
2023-09-28 23:28:28,017 - utils - INFO - 1, epoch: 879, all client loss: [0.5338090062141418, 0.47499704360961914], all pred client disparities: [0.010332942008972168, 0.008760079741477966], all client disparities: [0.015942007303237915, 0.012139573693275452], all client accs: [0.7651332020759583, 0.7827547788619995],  alphas:tensor([0.7501, 0.0000, 0.0000, 0.2499], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:28,146 - utils - INFO - valid: True, epoch: 879, loss: [0.5809136033058167, 0.4744049608707428], accuracy: [0.7292817831039429, 0.7853416204452515], mean_accuracy:0.7573117017745972,variance_accuracy:0.028029918670654297, disparity: [0.015151500701904297, 0.008958160877227783], mean_disparity:0.01205483078956604,variance_disparity:0.003096669912338257, pred_disparity: [0.018852680921554565, 0.00374642014503479]
2023-09-28 23:28:28,239 - utils - INFO - global_valid: True, epoch: 879,  global_loss: 0.4755890667438507, global_accuracy: 0.8104312471654476,  global_disparity:0.006067097187042236, global_pred_disparity: 0.0012044757604599,
2023-09-28 23:28:28,475 - utils - INFO - stage2_gradient_single_runtime: 0.006258487701416016
2023-09-28 23:28:28,480 - utils - INFO - 1, epoch: 880, all client loss: [0.5338882803916931, 0.4746784269809723], all pred client disparities: [0.01030886173248291, 0.008524760603904724], all client disparities: [0.015942007303237915, 0.011993393301963806], all client accs: [0.7651332020759583, 0.7829414010047913],  alphas:tensor([0.7441, 0.0000, 0.0000, 0.2559], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:28,713 - utils - INFO - stage2_gradient_single_runtime: 0.006279945373535156
2023-09-28 23:28:28,718 - utils - INFO - 1, epoch: 881, all client loss: [0.5339714884757996, 0.47436344623565674], all pred client disparities: [0.010280102491378784, 0.008287042379379272], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.7833457589149475],  alphas:tensor([0.7382, 0.0000, 0.0000, 0.2618], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:28,953 - utils - INFO - stage2_gradient_single_runtime: 0.006298542022705078
2023-09-28 23:28:28,958 - utils - INFO - 1, epoch: 882, all client loss: [0.534058690071106, 0.4740521311759949], all pred client disparities: [0.010246515274047852, 0.008046925067901611], all client disparities: [0.015942007303237915, 0.013570070266723633], all client accs: [0.7651332020759583, 0.7833457589149475],  alphas:tensor([0.4597, 0.0000, 0.0919, 0.4484], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:29,195 - utils - INFO - stage2_gradient_single_runtime: 0.0062999725341796875
2023-09-28 23:28:29,200 - utils - INFO - 1, epoch: 883, all client loss: [0.5341354012489319, 0.47412681579589844], all pred client disparities: [0.010104477405548096, 0.007916048169136047], all client disparities: [0.015942007303237915, 0.013350799679756165], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.4592, 0.0000, 0.0924, 0.4485], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:29,439 - utils - INFO - stage2_gradient_single_runtime: 0.006246328353881836
2023-09-28 23:28:29,444 - utils - INFO - 1, epoch: 884, all client loss: [0.5342140793800354, 0.47420066595077515], all pred client disparities: [0.009958714246749878, 0.007783800363540649], all client disparities: [0.015942007303237915, 0.013350799679756165], all client accs: [0.7651332020759583, 0.7834390997886658],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:29,684 - utils - INFO - stage2_gradient_single_runtime: 0.0064318180084228516
2023-09-28 23:28:29,689 - utils - INFO - 1, epoch: 885, all client loss: [0.5336482524871826, 0.4748673439025879], all pred client disparities: [0.010368764400482178, 0.008621826767921448], all client disparities: [0.015942007303237915, 0.012212663888931274], all client accs: [0.7651332020759583, 0.7826303243637085],  alphas:tensor([0.7518, 0.0000, 0.0000, 0.2482], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:29,927 - utils - INFO - stage2_gradient_single_runtime: 0.00623774528503418
2023-09-28 23:28:29,932 - utils - INFO - 1, epoch: 886, all client loss: [0.5337246656417847, 0.47455015778541565], all pred client disparities: [0.010349184274673462, 0.008389502763748169], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.7833146452903748],  alphas:tensor([0.7459, 0.0000, 0.0000, 0.2541], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:30,171 - utils - INFO - stage2_gradient_single_runtime: 0.006451129913330078
2023-09-28 23:28:30,177 - utils - INFO - 1, epoch: 887, all client loss: [0.5338048934936523, 0.4742366671562195], all pred client disparities: [0.010325253009796143, 0.00815485417842865], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.7833457589149475],  alphas:tensor([0.7399, 0.0000, 0.0000, 0.2601], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:30,416 - utils - INFO - stage2_gradient_single_runtime: 0.006266117095947266
2023-09-28 23:28:30,421 - utils - INFO - 1, epoch: 888, all client loss: [0.5338889956474304, 0.473926842212677], all pred client disparities: [0.010296612977981567, 0.007917910814285278], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.7834702134132385],  alphas:tensor([0.7341, 0.0000, 0.0000, 0.2659], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:30,659 - utils - INFO - stage2_gradient_single_runtime: 0.006727933883666992
2023-09-28 23:28:30,664 - utils - INFO - 1, epoch: 889, all client loss: [0.5339770317077637, 0.47362059354782104], all pred client disparities: [0.01026347279548645, 0.007678776979446411], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.4594, 0.0000, 0.0921, 0.4485], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:30,900 - utils - INFO - stage2_gradient_single_runtime: 0.006346940994262695
2023-09-28 23:28:30,905 - utils - INFO - 1, epoch: 890, all client loss: [0.534051775932312, 0.47369611263275146], all pred client disparities: [0.01012459397315979, 0.007550463080406189], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.4589, 0.0000, 0.0925, 0.4486], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:31,150 - utils - INFO - stage2_gradient_single_runtime: 0.00625920295715332
2023-09-28 23:28:31,154 - utils - INFO - 1, epoch: 891, all client loss: [0.5341284275054932, 0.4737709164619446], all pred client disparities: [0.009982109069824219, 0.00742088258266449], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:31,395 - utils - INFO - stage2_gradient_single_runtime: 0.006285905838012695
2023-09-28 23:28:31,399 - utils - INFO - 1, epoch: 892, all client loss: [0.5335656404495239, 0.47442880272865295], all pred client disparities: [0.01038256287574768, 0.008250236511230469], all client disparities: [0.015942007303237915, 0.011554867029190063], all client accs: [0.7651332020759583, 0.7831902503967285],  alphas:tensor([0.7476, 0.0000, 0.0000, 0.2524], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:31,636 - utils - INFO - stage2_gradient_single_runtime: 0.010557889938354492
2023-09-28 23:28:31,643 - utils - INFO - 1, epoch: 893, all client loss: [0.5336431264877319, 0.4741166830062866], all pred client disparities: [0.010363072156906128, 0.008018553256988525], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.7833146452903748],  alphas:tensor([0.7417, 0.0000, 0.0000, 0.2583], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:31,909 - utils - INFO - stage2_gradient_single_runtime: 0.00698089599609375
2023-09-28 23:28:31,914 - utils - INFO - 1, epoch: 894, all client loss: [0.5337242484092712, 0.47380825877189636], all pred client disparities: [0.01033926010131836, 0.007784664630889893], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.7834702134132385],  alphas:tensor([0.7359, 0.0000, 0.0000, 0.2641], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:32,197 - utils - INFO - stage2_gradient_single_runtime: 0.006439208984375
2023-09-28 23:28:32,202 - utils - INFO - 1, epoch: 895, all client loss: [0.5338091254234314, 0.473503440618515], all pred client disparities: [0.010311037302017212, 0.007548689842224121], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.4596, 0.0000, 0.0924, 0.4480], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:32,451 - utils - INFO - stage2_gradient_single_runtime: 0.006329059600830078
2023-09-28 23:28:32,456 - utils - INFO - 1, epoch: 896, all client loss: [0.5338809490203857, 0.47358015179634094], all pred client disparities: [0.01017722487449646, 0.00742340087890625], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.4591, 0.0000, 0.0928, 0.4481], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:32,689 - utils - INFO - stage2_gradient_single_runtime: 0.006319999694824219
2023-09-28 23:28:32,694 - utils - INFO - 1, epoch: 897, all client loss: [0.5339545011520386, 0.4736562669277191], all pred client disparities: [0.010039955377578735, 0.007296904921531677], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.4586, 0.0000, 0.0933, 0.4482], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:32,928 - utils - INFO - stage2_gradient_single_runtime: 0.006262302398681641
2023-09-28 23:28:32,933 - utils - INFO - 1, epoch: 898, all client loss: [0.5340297818183899, 0.47373166680336], all pred client disparities: [0.009899169206619263, 0.0071692317724227905], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:33,166 - utils - INFO - stage2_gradient_single_runtime: 0.006281852722167969
2023-09-28 23:28:33,171 - utils - INFO - 1, epoch: 899, all client loss: [0.5334721803665161, 0.4743862748146057], all pred client disparities: [0.010291993618011475, 0.007993444800376892], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.783252477645874],  alphas:tensor([0.7479, 0.0000, 0.0000, 0.2521], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:33,253 - utils - INFO - valid: True, epoch: 899, loss: [0.5808618068695068, 0.47379016876220703], accuracy: [0.7292817831039429, 0.7860248684883118], mean_accuracy:0.7576533257961273,variance_accuracy:0.028371542692184448, disparity: [0.015151500701904297, 0.008366450667381287], mean_disparity:0.011758975684642792,variance_disparity:0.003392525017261505, pred_disparity: [0.01806679368019104, 0.0031364858150482178]
2023-09-28 23:28:33,379 - utils - INFO - global_valid: True, epoch: 899,  global_loss: 0.47498053312301636, global_accuracy: 0.8103709858084712,  global_disparity:0.005494028329849243, global_pred_disparity: 0.0005699396133422852,
2023-09-28 23:28:33,616 - utils - INFO - stage2_gradient_single_runtime: 0.006308317184448242
2023-09-28 23:28:33,622 - utils - INFO - 1, epoch: 900, all client loss: [0.5335484147071838, 0.4740750789642334], all pred client disparities: [0.010274529457092285, 0.007763370871543884], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.783283531665802],  alphas:tensor([0.7420, 0.0000, 0.0000, 0.2580], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:33,911 - utils - INFO - stage2_gradient_single_runtime: 0.006333112716674805
2023-09-28 23:28:33,916 - utils - INFO - 1, epoch: 901, all client loss: [0.5336282253265381, 0.47376754879951477], all pred client disparities: [0.010252892971038818, 0.0075312405824661255], all client disparities: [0.015942007303237915, 0.01118941605091095], all client accs: [0.7651332020759583, 0.7835946083068848],  alphas:tensor([0.7362, 0.0000, 0.0000, 0.2638], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:34,151 - utils - INFO - stage2_gradient_single_runtime: 0.0063991546630859375
2023-09-28 23:28:34,156 - utils - INFO - 1, epoch: 902, all client loss: [0.5337117910385132, 0.4734637439250946], all pred client disparities: [0.010226845741271973, 0.00729706883430481], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.4593, 0.0000, 0.0932, 0.4476], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:34,393 - utils - INFO - stage2_gradient_single_runtime: 0.006330728530883789
2023-09-28 23:28:34,398 - utils - INFO - 1, epoch: 903, all client loss: [0.5337823033332825, 0.4735410809516907], all pred client disparities: [0.010094612836837769, 0.007173582911491394], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7834702134132385],  alphas:tensor([0.4588, 0.0000, 0.0936, 0.4476], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:34,632 - utils - INFO - stage2_gradient_single_runtime: 0.0062983036041259766
2023-09-28 23:28:34,637 - utils - INFO - 1, epoch: 904, all client loss: [0.5338544845581055, 0.4736177623271942], all pred client disparities: [0.009959042072296143, 0.007048994302749634], all client disparities: [0.015942007303237915, 0.013277709484100342], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:34,872 - utils - INFO - stage2_gradient_single_runtime: 0.006460428237915039
2023-09-28 23:28:34,877 - utils - INFO - 1, epoch: 905, all client loss: [0.5333032608032227, 0.4742693305015564], all pred client disparities: [0.01033744215965271, 0.007869645953178406], all client disparities: [0.015942007303237915, 0.011554867029190063], all client accs: [0.7651332020759583, 0.7831591367721558],  alphas:tensor([0.7501, 0.0000, 0.0000, 0.2499], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:35,111 - utils - INFO - stage2_gradient_single_runtime: 0.006241798400878906
2023-09-28 23:28:35,116 - utils - INFO - 1, epoch: 906, all client loss: [0.5333762764930725, 0.473959356546402], all pred client disparities: [0.010324805974960327, 0.007642820477485657], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.783283531665802],  alphas:tensor([0.7442, 0.0000, 0.0000, 0.2558], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:35,350 - utils - INFO - stage2_gradient_single_runtime: 0.006253719329833984
2023-09-28 23:28:35,356 - utils - INFO - 1, epoch: 907, all client loss: [0.5334529280662537, 0.4736531376838684], all pred client disparities: [0.010308176279067993, 0.007413968443870544], all client disparities: [0.015942007303237915, 0.01118941605091095], all client accs: [0.7651332020759583, 0.783563494682312],  alphas:tensor([0.7384, 0.0000, 0.0000, 0.2616], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:35,586 - utils - INFO - stage2_gradient_single_runtime: 0.006299734115600586
2023-09-28 23:28:35,591 - utils - INFO - 1, epoch: 908, all client loss: [0.5335330367088318, 0.47335055470466614], all pred client disparities: [0.010287374258041382, 0.00718313455581665], all client disparities: [0.015942007303237915, 0.013204619288444519], all client accs: [0.7651332020759583, 0.7836568355560303],  alphas:tensor([0.4595, 0.0000, 0.0935, 0.4470], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:35,825 - utils - INFO - stage2_gradient_single_runtime: 0.006274223327636719
2023-09-28 23:28:35,830 - utils - INFO - 1, epoch: 909, all client loss: [0.533600389957428, 0.47342923283576965], all pred client disparities: [0.0101604163646698, 0.007062837481498718], all client disparities: [0.015942007303237915, 0.014645546674728394], all client accs: [0.7651332020759583, 0.7836257219314575],  alphas:tensor([0.4591, 0.0000, 0.0939, 0.4471], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:36,059 - utils - INFO - stage2_gradient_single_runtime: 0.00628209114074707
2023-09-28 23:28:36,065 - utils - INFO - 1, epoch: 910, all client loss: [0.5336693525314331, 0.473507285118103], all pred client disparities: [0.010030388832092285, 0.006941467523574829], all client disparities: [0.015942007303237915, 0.014645546674728394], all client accs: [0.7651332020759583, 0.7833768725395203],  alphas:tensor([0.4586, 0.0000, 0.0942, 0.4472], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:36,305 - utils - INFO - stage2_gradient_single_runtime: 0.006247520446777344
2023-09-28 23:28:36,309 - utils - INFO - 1, epoch: 911, all client loss: [0.5337398648262024, 0.47358474135398865], all pred client disparities: [0.009897202253341675, 0.006819054484367371], all client disparities: [0.015942007303237915, 0.014718636870384216], all client accs: [0.7651332020759583, 0.783407986164093],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:36,542 - utils - INFO - stage2_gradient_single_runtime: 0.006222724914550781
2023-09-28 23:28:36,547 - utils - INFO - 1, epoch: 912, all client loss: [0.5331941843032837, 0.4742332696914673], all pred client disparities: [0.01026657223701477, 0.007634907960891724], all client disparities: [0.015942007303237915, 0.011554867029190063], all client accs: [0.7651332020759583, 0.7831591367721558],  alphas:tensor([0.7510, 0.0000, 0.0000, 0.2490], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:36,779 - utils - INFO - stage2_gradient_single_runtime: 0.006266117095947266
2023-09-28 23:28:36,784 - utils - INFO - 1, epoch: 913, all client loss: [0.5332654714584351, 0.47392407059669495], all pred client disparities: [0.010256558656692505, 0.007410138845443726], all client disparities: [0.015942007303237915, 0.011408686637878418], all client accs: [0.7651332020759583, 0.783283531665802],  alphas:tensor([0.7452, 0.0000, 0.0000, 0.2548], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:37,018 - utils - INFO - stage2_gradient_single_runtime: 0.006276845932006836
2023-09-28 23:28:37,023 - utils - INFO - 1, epoch: 914, all client loss: [0.5333402156829834, 0.47361862659454346], all pred client disparities: [0.010242730379104614, 0.00718340277671814], all client disparities: [0.015942007303237915, 0.01118941605091095], all client accs: [0.7651332020759583, 0.783563494682312],  alphas:tensor([0.7394, 0.0000, 0.0000, 0.2606], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:37,254 - utils - INFO - stage2_gradient_single_runtime: 0.0062618255615234375
2023-09-28 23:28:37,259 - utils - INFO - 1, epoch: 915, all client loss: [0.5334184765815735, 0.47331687808036804], all pred client disparities: [0.010224968194961548, 0.0069548338651657104], all client disparities: [0.015942007303237915, 0.014645546674728394], all client accs: [0.7651332020759583, 0.7835946083068848],  alphas:tensor([0.4594, 0.0000, 0.0941, 0.4465], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:37,503 - utils - INFO - stage2_gradient_single_runtime: 0.006345510482788086
2023-09-28 23:28:37,508 - utils - INFO - 1, epoch: 916, all client loss: [0.533484160900116, 0.47339633107185364], all pred client disparities: [0.010100245475769043, 0.0068366676568984985], all client disparities: [0.015942007303237915, 0.014645546674728394], all client accs: [0.7651332020759583, 0.7835946083068848],  alphas:tensor([0.4589, 0.0000, 0.0945, 0.4466], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:37,738 - utils - INFO - stage2_gradient_single_runtime: 0.0062732696533203125
2023-09-28 23:28:37,743 - utils - INFO - 1, epoch: 917, all client loss: [0.5335513949394226, 0.4734751582145691], all pred client disparities: [0.009972602128982544, 0.006717488169670105], all client disparities: [0.015942007303237915, 0.014645546674728394], all client accs: [0.7651332020759583, 0.7833768725395203],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:37,972 - utils - INFO - stage2_gradient_single_runtime: 0.006209611892700195
2023-09-28 23:28:37,977 - utils - INFO - 1, epoch: 918, all client loss: [0.5330122709274292, 0.4741208553314209], all pred client disparities: [0.010326594114303589, 0.007530137896537781], all client disparities: [0.015942007303237915, 0.011554867029190063], all client accs: [0.7651332020759583, 0.7831591367721558],  alphas:tensor([0.7538, 0.0000, 0.0000, 0.2462], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:38,210 - utils - INFO - stage2_gradient_single_runtime: 0.006293773651123047
2023-09-28 23:28:38,215 - utils - INFO - 1, epoch: 919, all client loss: [0.5330801606178284, 0.47381269931793213], all pred client disparities: [0.010321766138076782, 0.007308855652809143], all client disparities: [0.015942007303237915, 0.011554867029190063], all client accs: [0.7651332020759583, 0.7831902503967285],  alphas:tensor([0.7479, 0.0000, 0.0000, 0.2521], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:38,294 - utils - INFO - valid: True, epoch: 919, loss: [0.5807907581329346, 0.47321271896362305], accuracy: [0.7348066568374634, 0.7860869765281677], mean_accuracy:0.7604468166828156,variance_accuracy:0.025640159845352173, disparity: [0.019696980714797974, 0.008070588111877441], mean_disparity:0.013883784413337708,variance_disparity:0.005813196301460266, pred_disparity: [0.017193317413330078, 0.0026082247495651245]
2023-09-28 23:28:38,425 - utils - INFO - global_valid: True, epoch: 919,  global_loss: 0.47440868616104126, global_accuracy: 0.8104245247413435,  global_disparity:0.005350768566131592, global_pred_disparity: 1.2546777725219727e-05,
2023-09-28 23:28:38,661 - utils - INFO - stage2_gradient_single_runtime: 0.006249189376831055
2023-09-28 23:28:38,667 - utils - INFO - 1, epoch: 920, all client loss: [0.5331512689590454, 0.473508358001709], all pred client disparities: [0.010313302278518677, 0.007085695862770081], all client disparities: [0.015942007303237915, 0.01118941605091095], all client accs: [0.7651332020759583, 0.7835323810577393],  alphas:tensor([0.7421, 0.0000, 0.0000, 0.2579], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:38,910 - utils - INFO - stage2_gradient_single_runtime: 0.006272554397583008
2023-09-28 23:28:38,916 - utils - INFO - 1, epoch: 921, all client loss: [0.5332257747650146, 0.4732077121734619], all pred client disparities: [0.01030111312866211, 0.0068607330322265625], all client disparities: [0.015942007303237915, 0.012484163045883179], all client accs: [0.7651332020759583, 0.783687949180603],  alphas:tensor([0.4597, 0.0000, 0.0944, 0.4459], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:39,153 - utils - INFO - stage2_gradient_single_runtime: 0.006229400634765625
2023-09-28 23:28:39,159 - utils - INFO - 1, epoch: 922, all client loss: [0.5332880616188049, 0.47328856587409973], all pred client disparities: [0.01018187403678894, 0.006745919585227966], all client disparities: [0.015942007303237915, 0.014645546674728394], all client accs: [0.7651332020759583, 0.7835946083068848],  alphas:tensor([0.7352, 0.0000, 0.0000, 0.2648], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:39,397 - utils - INFO - stage2_gradient_single_runtime: 0.006290435791015625
2023-09-28 23:28:39,403 - utils - INFO - 1, epoch: 923, all client loss: [0.5333672761917114, 0.4729912579059601], all pred client disparities: [0.010163843631744385, 0.006518259644508362], all client disparities: [0.015942007303237915, 0.015574842691421509], all client accs: [0.7651332020759583, 0.7846211194992065],  alphas:tensor([0.4588, 0.0000, 0.0946, 0.4465], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:39,592 - utils - INFO - stage2_gradient_single_runtime: 0.006264686584472656
2023-09-28 23:28:39,599 - utils - INFO - 1, epoch: 924, all client loss: [0.5334316492080688, 0.4730713665485382], all pred client disparities: [0.010040700435638428, 0.006402462720870972], all client disparities: [0.015942007303237915, 0.015574842691421509], all client accs: [0.7651332020759583, 0.7845900058746338],  alphas:tensor([0.4584, 0.0000, 0.0950, 0.4466], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:39,843 - utils - INFO - stage2_gradient_single_runtime: 0.006250619888305664
2023-09-28 23:28:39,849 - utils - INFO - 1, epoch: 925, all client loss: [0.5334973931312561, 0.47315090894699097], all pred client disparities: [0.009914785623550415, 0.006285741925239563], all client disparities: [0.015942007303237915, 0.015574842691421509], all client accs: [0.7651332020759583, 0.7845900058746338],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:40,088 - utils - INFO - stage2_gradient_single_runtime: 0.006270885467529297
2023-09-28 23:28:40,094 - utils - INFO - 1, epoch: 926, all client loss: [0.5329614877700806, 0.4737885594367981], all pred client disparities: [0.01026192307472229, 0.007088899612426758], all client disparities: [0.015942007303237915, 0.011554867029190063], all client accs: [0.7651332020759583, 0.7831902503967285],  alphas:tensor([0.7494, 0.0000, 0.0000, 0.2506], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:40,334 - utils - INFO - stage2_gradient_single_runtime: 0.006333827972412109
2023-09-28 23:28:40,340 - utils - INFO - 1, epoch: 927, all client loss: [0.5330305099487305, 0.4734848141670227], all pred client disparities: [0.010256528854370117, 0.006868079304695129], all client disparities: [0.015942007303237915, 0.012630343437194824], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.7436, 0.0000, 0.0000, 0.2564], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:40,583 - utils - INFO - stage2_gradient_single_runtime: 0.006353616714477539
2023-09-28 23:28:40,590 - utils - INFO - 1, epoch: 928, all client loss: [0.533102810382843, 0.4731847643852234], all pred client disparities: [0.010247617959976196, 0.0066455453634262085], all client disparities: [0.015942007303237915, 0.014499381184577942], all client accs: [0.7651332020759583, 0.7837812304496765],  alphas:tensor([0.7379, 0.0000, 0.0000, 0.2621], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:40,827 - utils - INFO - stage2_gradient_single_runtime: 0.006325244903564453
2023-09-28 23:28:40,834 - utils - INFO - 1, epoch: 929, all client loss: [0.5331783294677734, 0.4728884696960449], all pred client disparities: [0.01023516058921814, 0.006421402096748352], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845900058746338],  alphas:tensor([0.4592, 0.0000, 0.0949, 0.4459], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:41,067 - utils - INFO - stage2_gradient_single_runtime: 0.006255626678466797
2023-09-28 23:28:41,074 - utils - INFO - 1, epoch: 930, all client loss: [0.5332394242286682, 0.4729699492454529], all pred client disparities: [0.01011723279953003, 0.006308794021606445], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.784558892250061],  alphas:tensor([0.4588, 0.0000, 0.0953, 0.4460], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:41,315 - utils - INFO - stage2_gradient_single_runtime: 0.006318807601928711
2023-09-28 23:28:41,318 - utils - INFO - 1, epoch: 931, all client loss: [0.533301830291748, 0.47305089235305786], all pred client disparities: [0.009996861219406128, 0.006195381283760071], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.784558892250061],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:41,576 - utils - INFO - stage2_gradient_single_runtime: 0.006502628326416016
2023-09-28 23:28:41,582 - utils - INFO - 1, epoch: 932, all client loss: [0.5327727794647217, 0.4736859202384949], all pred client disparities: [0.010328292846679688, 0.0069956183433532715], all client disparities: [0.015942007303237915, 0.011554867029190063], all client accs: [0.7651332020759583, 0.7831902503967285],  alphas:tensor([0.7525, 0.0000, 0.0000, 0.2475], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:41,825 - utils - INFO - stage2_gradient_single_runtime: 0.00627446174621582
2023-09-28 23:28:41,830 - utils - INFO - 1, epoch: 933, all client loss: [0.5328381061553955, 0.4733830392360687], all pred client disparities: [0.010328173637390137, 0.006778433453291655], all client disparities: [0.015942007303237915, 0.012630343437194824], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.7467, 0.0000, 0.0000, 0.2533], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:42,064 - utils - INFO - stage2_gradient_single_runtime: 0.006280660629272461
2023-09-28 23:28:42,069 - utils - INFO - 1, epoch: 934, all client loss: [0.5329065918922424, 0.4730839133262634], all pred client disparities: [0.010324805974960327, 0.0065595656633377075], all client disparities: [0.015942007303237915, 0.012264907360076904], all client accs: [0.7651332020759583, 0.783843457698822],  alphas:tensor([0.7410, 0.0000, 0.0000, 0.2590], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:42,309 - utils - INFO - stage2_gradient_single_runtime: 0.006309986114501953
2023-09-28 23:28:42,314 - utils - INFO - 1, epoch: 935, all client loss: [0.5329781174659729, 0.47278857231140137], all pred client disparities: [0.01031804084777832, 0.006339117884635925], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.784558892250061],  alphas:tensor([0.4597, 0.0000, 0.0951, 0.4452], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:42,552 - utils - INFO - stage2_gradient_single_runtime: 0.00651240348815918
2023-09-28 23:28:42,557 - utils - INFO - 1, epoch: 936, all client loss: [0.5330358147621155, 0.47287145256996155], all pred client disparities: [0.010205566883087158, 0.006229907274246216], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([0.4592, 0.0000, 0.0955, 0.4453], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:42,802 - utils - INFO - stage2_gradient_single_runtime: 0.006243228912353516
2023-09-28 23:28:42,807 - utils - INFO - 1, epoch: 937, all client loss: [0.5330946445465088, 0.4729539155960083], all pred client disparities: [0.010090827941894531, 0.006119921803474426], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([0.4588, 0.0000, 0.0959, 0.4453], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:43,044 - utils - INFO - stage2_gradient_single_runtime: 0.006419181823730469
2023-09-28 23:28:43,049 - utils - INFO - 1, epoch: 938, all client loss: [0.5331547856330872, 0.47303590178489685], all pred client disparities: [0.009973853826522827, 0.006009116768836975], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:43,296 - utils - INFO - stage2_gradient_single_runtime: 0.006256580352783203
2023-09-28 23:28:43,301 - utils - INFO - 1, epoch: 939, all client loss: [0.5326321721076965, 0.4736684262752533], all pred client disparities: [0.010293722152709961, 0.006805360317230225], all client disparities: [0.015942007303237915, 0.012995794415473938], all client accs: [0.7651332020759583, 0.783128023147583],  alphas:tensor([0.7548, 0.0000, 0.0000, 0.2452], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:43,429 - utils - INFO - valid: True, epoch: 939, loss: [0.5807896852493286, 0.4730733335018158], accuracy: [0.7348066568374634, 0.7861490845680237], mean_accuracy:0.7604778707027435,variance_accuracy:0.02567121386528015, disparity: [0.019696980714797974, 0.008070588111877441], mean_disparity:0.013883784413337708,variance_disparity:0.005813196301460266, pred_disparity: [0.016278326511383057, 0.0022322386503219604]
2023-09-28 23:28:43,514 - utils - INFO - global_valid: True, epoch: 939,  global_loss: 0.47427088022232056, global_accuracy: 0.8104604996423419,  global_disparity:0.005350768566131592, global_pred_disparity: 0.0003965795040130615,
2023-09-28 23:28:43,749 - utils - INFO - stage2_gradient_single_runtime: 0.006256580352783203
2023-09-28 23:28:43,756 - utils - INFO - 1, epoch: 940, all client loss: [0.5326948165893555, 0.4733659029006958], all pred client disparities: [0.010297417640686035, 0.006591007113456726], all client disparities: [0.015942007303237915, 0.012557253241539001], all client accs: [0.7651332020759583, 0.7835323810577393],  alphas:tensor([0.7491, 0.0000, 0.0000, 0.2509], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:43,995 - utils - INFO - stage2_gradient_single_runtime: 0.006876707077026367
2023-09-28 23:28:44,003 - utils - INFO - 1, epoch: 941, all client loss: [0.5327604413032532, 0.47306716442108154], all pred client disparities: [0.010297924280166626, 0.006375059485435486], all client disparities: [0.015942007303237915, 0.012264907360076904], all client accs: [0.7651332020759583, 0.783843457698822],  alphas:tensor([0.7434, 0.0000, 0.0000, 0.2566], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:44,257 - utils - INFO - stage2_gradient_single_runtime: 0.006569385528564453
2023-09-28 23:28:44,263 - utils - INFO - 1, epoch: 942, all client loss: [0.5328290462493896, 0.4727722108364105], all pred client disparities: [0.010295331478118896, 0.006157591938972473], all client disparities: [0.015942007303237915, 0.013340368866920471], all client accs: [0.7651332020759583, 0.7846211194992065],  alphas:tensor([0.4597, 0.0000, 0.0957, 0.4446], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:44,507 - utils - INFO - stage2_gradient_single_runtime: 0.006303071975708008
2023-09-28 23:28:44,511 - utils - INFO - 1, epoch: 943, all client loss: [0.5328844785690308, 0.4728561341762543], all pred client disparities: [0.010186105966567993, 0.006051063537597656], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([0.4593, 0.0000, 0.0961, 0.4446], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:44,749 - utils - INFO - stage2_gradient_single_runtime: 0.006474733352661133
2023-09-28 23:28:44,755 - utils - INFO - 1, epoch: 944, all client loss: [0.5329408645629883, 0.4729396104812622], all pred client disparities: [0.010074913501739502, 0.005943745374679565], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([0.4589, 0.0000, 0.0964, 0.4447], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:44,989 - utils - INFO - stage2_gradient_single_runtime: 0.006300210952758789
2023-09-28 23:28:44,996 - utils - INFO - 1, epoch: 945, all client loss: [0.5329985618591309, 0.4730226695537567], all pred client disparities: [0.009961575269699097, 0.00583571195602417], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:45,233 - utils - INFO - stage2_gradient_single_runtime: 0.006270170211791992
2023-09-28 23:28:45,240 - utils - INFO - 1, epoch: 946, all client loss: [0.5328457951545715, 0.47276943922042847], all pred client disparities: [0.010175347328186035, 0.005921497475355864], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([0.4592, 0.0000, 0.0963, 0.4445], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:45,481 - utils - INFO - stage2_gradient_single_runtime: 0.0064601898193359375
2023-09-28 23:28:45,487 - utils - INFO - 1, epoch: 947, all client loss: [0.5329015254974365, 0.47285327315330505], all pred client disparities: [0.01006510853767395, 0.005815252661705017], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([0.4588, 0.0000, 0.0966, 0.4446], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:45,724 - utils - INFO - stage2_gradient_single_runtime: 0.006266117095947266
2023-09-28 23:28:45,730 - utils - INFO - 1, epoch: 948, all client loss: [0.5329583883285522, 0.4729367196559906], all pred client disparities: [0.009952902793884277, 0.005708277225494385], all client disparities: [0.015942007303237915, 0.015501752495765686], all client accs: [0.7651332020759583, 0.7845277786254883],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:45,967 - utils - INFO - stage2_gradient_single_runtime: 0.006230831146240234
2023-09-28 23:28:45,973 - utils - INFO - 1, epoch: 949, all client loss: [0.5324443578720093, 0.47356459498405457], all pred client disparities: [0.010256737470626831, 0.006497874855995178], all client disparities: [0.015942007303237915, 0.01277652382850647], all client accs: [0.7651332020759583, 0.7833146452903748],  alphas:tensor([0.7570, 0.0000, 0.0000, 0.2430], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:46,215 - utils - INFO - stage2_gradient_single_runtime: 0.006963014602661133
2023-09-28 23:28:46,222 - utils - INFO - 1, epoch: 950, all client loss: [0.5325037240982056, 0.4732634127140045], all pred client disparities: [0.01026502251625061, 0.0062872618436813354], all client disparities: [0.015942007303237915, 0.012411072850227356], all client accs: [0.7651332020759583, 0.7834702134132385],  alphas:tensor([0.7512, 0.0000, 0.0000, 0.2488], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:46,457 - utils - INFO - stage2_gradient_single_runtime: 0.0062596797943115234
2023-09-28 23:28:46,463 - utils - INFO - 1, epoch: 951, all client loss: [0.5325659513473511, 0.47296610474586487], all pred client disparities: [0.010270416736602783, 0.0060750991106033325], all client disparities: [0.015942007303237915, 0.012264907360076904], all client accs: [0.7651332020759583, 0.783843457698822],  alphas:tensor([0.7455, 0.0000, 0.0000, 0.2545], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:46,711 - utils - INFO - stage2_gradient_single_runtime: 0.006371736526489258
2023-09-28 23:28:46,718 - utils - INFO - 1, epoch: 952, all client loss: [0.5326310992240906, 0.47267255187034607], all pred client disparities: [0.010272890329360962, 0.005861535668373108], all client disparities: [0.015942007303237915, 0.013340368866920471], all client accs: [0.7651332020759583, 0.7846211194992065],  alphas:tensor([0.7399, 0.0000, 0.0000, 0.2601], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:46,950 - utils - INFO - stage2_gradient_single_runtime: 0.006336688995361328
2023-09-28 23:28:46,957 - utils - INFO - 1, epoch: 953, all client loss: [0.5326990485191345, 0.4723827838897705], all pred client disparities: [0.010272353887557983, 0.0056466758251190186], all client disparities: [0.015942007303237915, 0.015428662300109863], all client accs: [0.7651332020759583, 0.7845900058746338],  alphas:tensor([0.4594, 0.0000, 0.0963, 0.4443], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:47,195 - utils - INFO - stage2_gradient_single_runtime: 0.006593465805053711
2023-09-28 23:28:47,202 - utils - INFO - 1, epoch: 954, all client loss: [0.5327513813972473, 0.47246816754341125], all pred client disparities: [0.010166972875595093, 0.00554424524307251], all client disparities: [0.015942007303237915, 0.015428662300109863], all client accs: [0.7651332020759583, 0.7845900058746338],  alphas:tensor([0.4590, 0.0000, 0.0967, 0.4443], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:47,440 - utils - INFO - stage2_gradient_single_runtime: 0.007490873336791992
2023-09-28 23:28:47,446 - utils - INFO - 1, epoch: 955, all client loss: [0.5328048467636108, 0.4725530743598938], all pred client disparities: [0.010059744119644165, 0.005441159009933472], all client disparities: [0.015942007303237915, 0.015428662300109863], all client accs: [0.7651332020759583, 0.7845900058746338],  alphas:tensor([0.4586, 0.0000, 0.0970, 0.4444], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:47,696 - utils - INFO - stage2_gradient_single_runtime: 0.0062787532806396484
2023-09-28 23:28:47,701 - utils - INFO - 1, epoch: 956, all client loss: [0.5328592658042908, 0.4726375937461853], all pred client disparities: [0.009950518608093262, 0.0053374022245407104], all client disparities: [0.015942007303237915, 0.013236001133918762], all client accs: [0.7651332020759583, 0.7843100428581238],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:47,938 - utils - INFO - stage2_gradient_single_runtime: 0.0062372684478759766
2023-09-28 23:28:47,943 - utils - INFO - 1, epoch: 957, all client loss: [0.5323496460914612, 0.47325825691223145], all pred client disparities: [0.010243922472000122, 0.0061184316873550415], all client disparities: [0.015942007303237915, 0.012484163045883179], all client accs: [0.7651332020759583, 0.7834390997886658],  alphas:tensor([0.7542, 0.0000, 0.0000, 0.2458], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:48,182 - utils - INFO - stage2_gradient_single_runtime: 0.006211519241333008
2023-09-28 23:28:48,188 - utils - INFO - 1, epoch: 958, all client loss: [0.5324086546897888, 0.4729610085487366], all pred client disparities: [0.010253489017486572, 0.00590941309928894], all client disparities: [0.015942007303237915, 0.012264907360076904], all client accs: [0.7651332020759583, 0.783843457698822],  alphas:tensor([0.7486, 0.0000, 0.0000, 0.2514], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:48,423 - utils - INFO - stage2_gradient_single_runtime: 0.006966829299926758
2023-09-28 23:28:48,429 - utils - INFO - 1, epoch: 959, all client loss: [0.5324705839157104, 0.4726675748825073], all pred client disparities: [0.010260343551635742, 0.005699083209037781], all client disparities: [0.015942007303237915, 0.013340368866920471], all client accs: [0.7651332020759583, 0.7846211194992065],  alphas:tensor([0.7429, 0.0000, 0.0000, 0.2571], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:48,510 - utils - INFO - valid: True, epoch: 959, loss: [0.5807302594184875, 0.47205671668052673], accuracy: [0.7348066568374634, 0.7866459488868713], mean_accuracy:0.7607263028621674,variance_accuracy:0.02591964602470398, disparity: [0.019696980714797974, 0.007035091519355774], mean_disparity:0.013366036117076874,variance_disparity:0.0063309445977211, pred_disparity: [0.01551768183708191, 0.0012943148612976074]
2023-09-28 23:28:48,641 - utils - INFO - global_valid: True, epoch: 959,  global_loss: 0.4732649028301239, global_accuracy: 0.8105089680064886,  global_disparity:0.004347890615463257, global_pred_disparity: 0.0013529807329177856,
2023-09-28 23:28:48,875 - utils - INFO - stage2_gradient_single_runtime: 0.006371259689331055
2023-09-28 23:28:48,882 - utils - INFO - 1, epoch: 960, all client loss: [0.532535195350647, 0.4723779559135437], all pred client disparities: [0.010264307260513306, 0.0054874420166015625], all client disparities: [0.015942007303237915, 0.015428662300109863], all client accs: [0.7651332020759583, 0.7845900058746338],  alphas:tensor([0.4596, 0.0000, 0.0968, 0.4435], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:49,115 - utils - INFO - stage2_gradient_single_runtime: 0.0062100887298583984
2023-09-28 23:28:49,121 - utils - INFO - 1, epoch: 961, all client loss: [0.532585084438324, 0.472464382648468], all pred client disparities: [0.010162442922592163, 0.005387783050537109], all client disparities: [0.015942007303237915, 0.013236001133918762], all client accs: [0.7651332020759583, 0.7843100428581238],  alphas:tensor([0.4592, 0.0000, 0.0972, 0.4436], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:49,356 - utils - INFO - stage2_gradient_single_runtime: 0.006383180618286133
2023-09-28 23:28:49,364 - utils - INFO - 1, epoch: 962, all client loss: [0.5326359272003174, 0.4725504219532013], all pred client disparities: [0.010058850049972534, 0.0052875131368637085], all client disparities: [0.015942007303237915, 0.013236001133918762], all client accs: [0.7651332020759583, 0.7843100428581238],  alphas:tensor([0.4589, 0.0000, 0.0975, 0.4436], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:49,606 - utils - INFO - stage2_gradient_single_runtime: 0.006250858306884766
2023-09-28 23:28:49,613 - utils - INFO - 1, epoch: 963, all client loss: [0.532687783241272, 0.4726361334323883], all pred client disparities: [0.009953528642654419, 0.005186617374420166], all client disparities: [0.015942007303237915, 0.013236001133918762], all client accs: [0.7651332020759583, 0.784278929233551],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:49,844 - utils - INFO - stage2_gradient_single_runtime: 0.0068628787994384766
2023-09-28 23:28:49,847 - utils - INFO - 1, epoch: 964, all client loss: [0.5321851968765259, 0.4732547700405121], all pred client disparities: [0.010233849287033081, 0.005964353680610657], all client disparities: [0.015942007303237915, 0.012630343437194824], all client accs: [0.7651332020759583, 0.7835013270378113],  alphas:tensor([0.7577, 0.0000, 0.0000, 0.2423], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:50,091 - utils - INFO - stage2_gradient_single_runtime: 0.0062868595123291016
2023-09-28 23:28:50,098 - utils - INFO - 1, epoch: 965, all client loss: [0.5322409272193909, 0.47295743227005005], all pred client disparities: [0.010247766971588135, 0.005758717656135559], all client disparities: [0.015942007303237915, 0.012411072850227356], all client accs: [0.7651332020759583, 0.7837190628051758],  alphas:tensor([0.7521, 0.0000, 0.0000, 0.2479], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:50,343 - utils - INFO - stage2_gradient_single_runtime: 0.006272792816162109
2023-09-28 23:28:50,350 - utils - INFO - 1, epoch: 966, all client loss: [0.5322992205619812, 0.4726639688014984], all pred client disparities: [0.010259121656417847, 0.005551725625991821], all client disparities: [0.015942007303237915, 0.013267278671264648], all client accs: [0.7651332020759583, 0.7845900058746338],  alphas:tensor([0.7464, 0.0000, 0.0000, 0.2536], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:50,581 - utils - INFO - stage2_gradient_single_runtime: 0.006264209747314453
2023-09-28 23:28:50,587 - utils - INFO - 1, epoch: 967, all client loss: [0.532360315322876, 0.4723743200302124], all pred client disparities: [0.010267853736877441, 0.005343541502952576], all client disparities: [0.015942007303237915, 0.011001527309417725], all client accs: [0.7651332020759583, 0.7844656109809875],  alphas:tensor([0.7409, 0.0000, 0.0000, 0.2591], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:50,820 - utils - INFO - stage2_gradient_single_runtime: 0.006644010543823242
2023-09-28 23:28:50,827 - utils - INFO - 1, epoch: 968, all client loss: [0.5324240326881409, 0.47208845615386963], all pred client disparities: [0.010273873805999756, 0.005134224891662598], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856165170669556],  alphas:tensor([0.4595, 0.0000, 0.0972, 0.4432], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:51,065 - utils - INFO - stage2_gradient_single_runtime: 0.006280183792114258
2023-09-28 23:28:51,072 - utils - INFO - 1, epoch: 969, all client loss: [0.5324712991714478, 0.47217610478401184], all pred client disparities: [0.010175347328186035, 0.005037903785705566], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856165170669556],  alphas:tensor([0.4592, 0.0000, 0.0976, 0.4433], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:51,308 - utils - INFO - stage2_gradient_single_runtime: 0.0062639713287353516
2023-09-28 23:28:51,315 - utils - INFO - 1, epoch: 970, all client loss: [0.5325194001197815, 0.4722634255886078], all pred client disparities: [0.010075122117996216, 0.004941076040267944], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856165170669556],  alphas:tensor([0.4588, 0.0000, 0.0979, 0.4433], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:51,557 - utils - INFO - stage2_gradient_single_runtime: 0.007527351379394531
2023-09-28 23:28:51,563 - utils - INFO - 1, epoch: 971, all client loss: [0.5325685739517212, 0.4723503291606903], all pred client disparities: [0.009973376989364624, 0.004843667149543762], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856165170669556],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:51,800 - utils - INFO - stage2_gradient_single_runtime: 0.006439924240112305
2023-09-28 23:28:51,804 - utils - INFO - 1, epoch: 972, all client loss: [0.5320708751678467, 0.4729621708393097], all pred client disparities: [0.010241806507110596, 0.005613371729850769], all client disparities: [0.015942007303237915, 0.012557253241539001], all client accs: [0.7651332020759583, 0.7837812304496765],  alphas:tensor([0.7559, 0.0000, 0.0000, 0.2441], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:52,038 - utils - INFO - stage2_gradient_single_runtime: 0.007645368576049805
2023-09-28 23:28:52,044 - utils - INFO - 1, epoch: 973, all client loss: [0.53212571144104, 0.47266849875450134], all pred client disparities: [0.010257601737976074, 0.00540982186794281], all client disparities: [0.015942007303237915, 0.010145336389541626], all client accs: [0.7651332020759583, 0.783563494682312],  alphas:tensor([0.7502, 0.0000, 0.0000, 0.2498], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:52,284 - utils - INFO - stage2_gradient_single_runtime: 0.007124900817871094
2023-09-28 23:28:52,291 - utils - INFO - 1, epoch: 974, all client loss: [0.532183051109314, 0.472378671169281], all pred client disparities: [0.010271012783050537, 0.005205094814300537], all client disparities: [0.015942007303237915, 0.010563001036643982], all client accs: [0.7651332020759583, 0.7857098579406738],  alphas:tensor([0.7447, 0.0000, 0.0000, 0.2553], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:52,533 - utils - INFO - stage2_gradient_single_runtime: 0.007350444793701172
2023-09-28 23:28:52,537 - utils - INFO - 1, epoch: 975, all client loss: [0.532243013381958, 0.4720926284790039], all pred client disparities: [0.010281950235366821, 0.004999279975891113], all client disparities: [0.015942007303237915, 0.01048991084098816], all client accs: [0.7651332020759583, 0.7857098579406738],  alphas:tensor([0.4599, 0.0000, 0.0977, 0.4424], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:52,781 - utils - INFO - stage2_gradient_single_runtime: 0.0068111419677734375
2023-09-28 23:28:52,784 - utils - INFO - 1, epoch: 976, all client loss: [0.53228759765625, 0.47218140959739685], all pred client disparities: [0.010187119245529175, 0.004905879497528076], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856165170669556],  alphas:tensor([0.4595, 0.0000, 0.0980, 0.4424], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:53,016 - utils - INFO - stage2_gradient_single_runtime: 0.00710296630859375
2023-09-28 23:28:53,022 - utils - INFO - 1, epoch: 977, all client loss: [0.5323330760002136, 0.4722699522972107], all pred client disparities: [0.010090947151184082, 0.004811972379684448], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856165170669556],  alphas:tensor([0.4592, 0.0000, 0.0984, 0.4425], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:53,262 - utils - INFO - stage2_gradient_single_runtime: 0.007095813751220703
2023-09-28 23:28:53,264 - utils - INFO - 1, epoch: 978, all client loss: [0.5323793292045593, 0.4723581075668335], all pred client disparities: [0.009993314743041992, 0.004717573523521423], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7855854034423828],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:53,507 - utils - INFO - stage2_gradient_single_runtime: 0.00715184211730957
2023-09-28 23:28:53,511 - utils - INFO - 1, epoch: 979, all client loss: [0.5318893194198608, 0.4729682207107544], all pred client disparities: [0.010247588157653809, 0.005484536290168762], all client disparities: [0.015942007303237915, 0.008203208446502686], all client accs: [0.7651332020759583, 0.7835323810577393],  alphas:tensor([0.7602, 0.0000, 0.0000, 0.2398], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:53,591 - utils - INFO - valid: True, epoch: 979, loss: [0.5807744264602661, 0.4723704755306244], accuracy: [0.7348066568374634, 0.7856521606445312], mean_accuracy:0.7602294087409973,variance_accuracy:0.025422751903533936, disparity: [0.019696980714797974, 0.004520297050476074], mean_disparity:0.012108638882637024,variance_disparity:0.00758834183216095, pred_disparity: [0.014443010091781616, 0.0011880993843078613]
2023-09-28 23:28:53,731 - utils - INFO - global_valid: True, epoch: 979,  global_loss: 0.4735756814479828, global_accuracy: 0.8103963281973177,  global_disparity:0.0019123703241348267, global_pred_disparity: 0.001502498984336853,
2023-09-28 23:28:53,973 - utils - INFO - stage2_gradient_single_runtime: 0.006729841232299805
2023-09-28 23:28:53,980 - utils - INFO - 1, epoch: 980, all client loss: [0.5319402813911438, 0.4726741313934326], all pred client disparities: [0.010268151760101318, 0.005284637212753296], all client disparities: [0.015942007303237915, 0.008130118250846863], all client accs: [0.7651332020759583, 0.783563494682312],  alphas:tensor([0.7546, 0.0000, 0.0000, 0.2454], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:54,246 - utils - INFO - stage2_gradient_single_runtime: 0.006443500518798828
2023-09-28 23:28:54,251 - utils - INFO - 1, epoch: 981, all client loss: [0.5319936871528625, 0.47238394618034363], all pred client disparities: [0.0102863609790802, 0.005083590745925903], all client disparities: [0.015942007303237915, 0.010636091232299805], all client accs: [0.7651332020759583, 0.7857098579406738],  alphas:tensor([0.7490, 0.0000, 0.0000, 0.2510], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:54,494 - utils - INFO - stage2_gradient_single_runtime: 0.0062754154205322266
2023-09-28 23:28:54,500 - utils - INFO - 1, epoch: 982, all client loss: [0.5320495963096619, 0.47209757566452026], all pred client disparities: [0.01030227541923523, 0.004881501197814941], all client disparities: [0.015942007303237915, 0.010636091232299805], all client accs: [0.7651332020759583, 0.7857720255851746],  alphas:tensor([0.7435, 0.0000, 0.0000, 0.2565], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:54,734 - utils - INFO - stage2_gradient_single_runtime: 0.006407737731933594
2023-09-28 23:28:54,740 - utils - INFO - 1, epoch: 983, all client loss: [0.5321080088615417, 0.4718150496482849], all pred client disparities: [0.010315924882888794, 0.0046784281730651855], all client disparities: [0.015942007303237915, 0.010636091232299805], all client accs: [0.7651332020759583, 0.7858031392097473],  alphas:tensor([0.4600, 0.0000, 0.0980, 0.4420], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:54,975 - utils - INFO - stage2_gradient_single_runtime: 0.006380796432495117
2023-09-28 23:28:54,981 - utils - INFO - 1, epoch: 984, all client loss: [0.532149612903595, 0.4719052016735077], all pred client disparities: [0.010224759578704834, 0.004588663578033447], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856476306915283],  alphas:tensor([0.4597, 0.0000, 0.0983, 0.4420], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:55,222 - utils - INFO - stage2_gradient_single_runtime: 0.006253957748413086
2023-09-28 23:28:55,228 - utils - INFO - 1, epoch: 985, all client loss: [0.5321919918060303, 0.471995085477829], all pred client disparities: [0.010132372379302979, 0.004498466849327087], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856476306915283],  alphas:tensor([0.4593, 0.0000, 0.0987, 0.4420], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:55,523 - utils - INFO - stage2_gradient_single_runtime: 0.006272315979003906
2023-09-28 23:28:55,529 - utils - INFO - 1, epoch: 986, all client loss: [0.5322352051734924, 0.4720846712589264], all pred client disparities: [0.010038644075393677, 0.004407837986946106], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7856165170669556],  alphas:tensor([0.4590, 0.0000, 0.0990, 0.4420], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:55,765 - utils - INFO - stage2_gradient_single_runtime: 0.006270647048950195
2023-09-28 23:28:55,770 - utils - INFO - 1, epoch: 987, all client loss: [0.5322791934013367, 0.4721739888191223], all pred client disparities: [0.009943783283233643, 0.004316732287406921], all client disparities: [0.015942007303237915, 0.012651294469833374], all client accs: [0.7651332020759583, 0.7857098579406738],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:56,005 - utils - INFO - stage2_gradient_single_runtime: 0.0062770843505859375
2023-09-28 23:28:56,011 - utils - INFO - 1, epoch: 988, all client loss: [0.5317946076393127, 0.47277799248695374], all pred client disparities: [0.01018783450126648, 0.00507509708404541], all client disparities: [0.015942007303237915, 0.007618501782417297], all client accs: [0.7651332020759583, 0.7848077416419983],  alphas:tensor([0.7587, 0.0000, 0.0000, 0.2413], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:56,249 - utils - INFO - stage2_gradient_single_runtime: 0.0063054561614990234
2023-09-28 23:28:56,254 - utils - INFO - 1, epoch: 989, all client loss: [0.5318445563316345, 0.4724866449832916], all pred client disparities: [0.0102100670337677, 0.004877358675003052], all client disparities: [0.015942007303237915, 0.008620887994766235], all client accs: [0.7651332020759583, 0.7857098579406738],  alphas:tensor([0.7531, 0.0000, 0.0000, 0.2469], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:56,492 - utils - INFO - stage2_gradient_single_runtime: 0.006245613098144531
2023-09-28 23:28:56,497 - utils - INFO - 1, epoch: 990, all client loss: [0.5318969488143921, 0.4721992611885071], all pred client disparities: [0.010230153799057007, 0.00467856228351593], all client disparities: [0.015942007303237915, 0.010636091232299805], all client accs: [0.7651332020759583, 0.7857098579406738],  alphas:tensor([0.7476, 0.0000, 0.0000, 0.2524], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:56,738 - utils - INFO - stage2_gradient_single_runtime: 0.0063860416412353516
2023-09-28 23:28:56,742 - utils - INFO - 1, epoch: 991, all client loss: [0.5319517254829407, 0.47191575169563293], all pred client disparities: [0.010247945785522461, 0.004478871822357178], all client disparities: [0.015942007303237915, 0.010636091232299805], all client accs: [0.7651332020759583, 0.7857720255851746],  alphas:tensor([0.4602, 0.0000, 0.0988, 0.4411], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:57,009 - utils - INFO - stage2_gradient_single_runtime: 0.010466814041137695
2023-09-28 23:28:57,015 - utils - INFO - 1, epoch: 992, all client loss: [0.5319913029670715, 0.4720068573951721], all pred client disparities: [0.010159462690353394, 0.004391655325889587], all client disparities: [0.015942007303237915, 0.01279747486114502], all client accs: [0.7651332020759583, 0.7856787443161011],  alphas:tensor([0.7417, 0.0000, 0.0000, 0.2583], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:57,267 - utils - INFO - stage2_gradient_single_runtime: 0.010576009750366211
2023-09-28 23:28:57,273 - utils - INFO - 1, epoch: 993, all client loss: [0.5320488810539246, 0.471726655960083], all pred client disparities: [0.010174423456192017, 0.004190891981124878], all client disparities: [0.015942007303237915, 0.012724384665489197], all client accs: [0.7651332020759583, 0.7857720255851746],  alphas:tensor([0.4595, 0.0000, 0.0990, 0.4416], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:57,546 - utils - INFO - stage2_gradient_single_runtime: 0.007348060607910156
2023-09-28 23:28:57,551 - utils - INFO - 1, epoch: 994, all client loss: [0.5320889353752136, 0.47181767225265503], all pred client disparities: [0.010084599256515503, 0.004103988409042358], all client disparities: [0.015942007303237915, 0.012578204274177551], all client accs: [0.7651332020759583, 0.7856787443161011],  alphas:tensor([0.4592, 0.0000, 0.0993, 0.4416], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:57,837 - utils - INFO - stage2_gradient_single_runtime: 0.006265401840209961
2023-09-28 23:28:57,844 - utils - INFO - 1, epoch: 995, all client loss: [0.5321297645568848, 0.47190842032432556], all pred client disparities: [0.009993672370910645, 0.00401674211025238], all client disparities: [0.015942007303237915, 0.002864152193069458], all client accs: [0.7651332020759583, 0.7762535810470581],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:58,136 - utils - INFO - stage2_gradient_single_runtime: 0.006231784820556641
2023-09-28 23:28:58,139 - utils - INFO - 1, epoch: 996, all client loss: [0.5316508412361145, 0.47250625491142273], all pred client disparities: [0.010223954916000366, 0.004767909646034241], all client disparities: [0.015942007303237915, 0.008620887994766235], all client accs: [0.7651332020759583, 0.7856787443161011],  alphas:tensor([0.7582, 0.0000, 0.0000, 0.2418], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:58,405 - utils - INFO - stage2_gradient_single_runtime: 0.010455131530761719
2023-09-28 23:28:58,411 - utils - INFO - 1, epoch: 997, all client loss: [0.5316991209983826, 0.4722181260585785], all pred client disparities: [0.010248959064483643, 0.0045728981494903564], all client disparities: [0.015942007303237915, 0.00847470760345459], all client accs: [0.7651332020759583, 0.7858031392097473],  alphas:tensor([0.7526, 0.0000, 0.0000, 0.2474], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:58,668 - utils - INFO - stage2_gradient_single_runtime: 0.008149385452270508
2023-09-28 23:28:58,673 - utils - INFO - 1, epoch: 998, all client loss: [0.5317496657371521, 0.47193387150764465], all pred client disparities: [0.010271787643432617, 0.004376992583274841], all client disparities: [0.015942007303237915, 0.008401617407798767], all client accs: [0.7651332020759583, 0.7858342528343201],  alphas:tensor([0.7471, 0.0000, 0.0000, 0.2529], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:58,925 - utils - INFO - stage2_gradient_single_runtime: 0.006266117095947266
2023-09-28 23:28:58,930 - utils - INFO - 1, epoch: 999, all client loss: [0.5318025350570679, 0.4716535210609436], all pred client disparities: [0.010292649269104004, 0.004180237650871277], all client disparities: [0.015942007303237915, 0.008401617407798767], all client accs: [0.7651332020759583, 0.7858964800834656],  alphas:tensor([0.4604, 0.0000, 0.0991, 0.4405], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:59,059 - utils - INFO - valid: True, epoch: 999, loss: [0.5807501077651978, 0.47141316533088684], accuracy: [0.7348066568374634, 0.7867701649665833], mean_accuracy:0.7607884109020233,variance_accuracy:0.025981754064559937, disparity: [0.019696980714797974, 0.002597227692604065], mean_disparity:0.01114710420370102,variance_disparity:0.008549876511096954, pred_disparity: [0.013655692338943481, 0.0001592785120010376]
2023-09-28 23:28:59,138 - utils - INFO - global_valid: True, epoch: 999,  global_loss: 0.47262874245643616, global_accuracy: 0.8103820783309821,  global_disparity:4.990399247617461e-05, global_pred_disparity: 0.0025495141744613647,
2023-09-28 23:28:59,370 - utils - INFO - stage2_gradient_single_runtime: 0.006243228912353516
2023-09-28 23:28:59,376 - utils - INFO - 1, epoch: 1000, all client loss: [0.5318390130996704, 0.471746027469635], all pred client disparities: [0.010207772254943848, 0.004096776247024536], all client disparities: [0.015942007303237915, 0.010563001036643982], all client accs: [0.7651332020759583, 0.7858031392097473],  alphas:tensor([0.4601, 0.0000, 0.0994, 0.4405], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:59,606 - utils - INFO - stage2_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:28:59,611 - utils - INFO - 1, epoch: 1001, all client loss: [0.5318762063980103, 0.4718382954597473], all pred client disparities: [0.010122090578079224, 0.004012897610664368], all client disparities: [0.015942007303237915, 0.012724384665489197], all client accs: [0.7651332020759583, 0.7857098579406738],  alphas:tensor([0.4598, 0.0000, 0.0996, 0.4405], device='cuda:0', dtype=torch.float64)
2023-09-28 23:28:59,845 - utils - INFO - stage2_gradient_single_runtime: 0.00625157356262207
2023-09-28 23:28:59,850 - utils - INFO - 1, epoch: 1002, all client loss: [0.5319139957427979, 0.47193029522895813], all pred client disparities: [0.010035306215286255, 0.003928706049919128], all client disparities: [0.015942007303237915, 0.0027179718017578125], all client accs: [0.7651332020759583, 0.7763158082962036],  alphas:tensor([0.7407, 0.0000, 0.0000, 0.2593], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:00,088 - utils - INFO - stage2_gradient_single_runtime: 0.006251335144042969
2023-09-28 23:29:00,093 - utils - INFO - 1, epoch: 1003, all client loss: [0.531970202922821, 0.47165217995643616], all pred client disparities: [0.010052233934402466, 0.0037306398153305054], all client disparities: [0.015942007303237915, 0.0029372423887252808], all client accs: [0.7651332020759583, 0.7766268253326416],  alphas:tensor([0.4591, 0.0000, 0.0999, 0.4410], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:00,331 - utils - INFO - stage2_gradient_single_runtime: 0.006206035614013672
2023-09-28 23:29:00,336 - utils - INFO - 1, epoch: 1004, all client loss: [0.5320082902908325, 0.47174420952796936], all pred client disparities: [0.009964317083358765, 0.0036469697952270508], all client disparities: [0.015942007303237915, 0.0029372423887252808], all client accs: [0.7651332020759583, 0.7766268253326416],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:00,570 - utils - INFO - stage2_gradient_single_runtime: 0.006216764450073242
2023-09-28 23:29:00,575 - utils - INFO - 1, epoch: 1005, all client loss: [0.5315354466438293, 0.47233638167381287], all pred client disparities: [0.010183095932006836, 0.0043900758028030396], all client disparities: [0.015942007303237915, 0.008693963289260864], all client accs: [0.7651332020759583, 0.7857098579406738],  alphas:tensor([0.7577, 0.0000, 0.0000, 0.2423], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:00,810 - utils - INFO - stage2_gradient_single_runtime: 0.006233930587768555
2023-09-28 23:29:00,815 - utils - INFO - 1, epoch: 1006, all client loss: [0.5315821170806885, 0.47205063700675964], all pred client disparities: [0.010210275650024414, 0.004197761416435242], all client disparities: [0.015942007303237915, 0.008401617407798767], all client accs: [0.7651332020759583, 0.7857720255851746],  alphas:tensor([0.7522, 0.0000, 0.0000, 0.2478], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:01,052 - utils - INFO - stage2_gradient_single_runtime: 0.0062334537506103516
2023-09-28 23:29:01,057 - utils - INFO - 1, epoch: 1007, all client loss: [0.5316309928894043, 0.471768856048584], all pred client disparities: [0.010235577821731567, 0.004004612565040588], all client disparities: [0.015942007303237915, 0.008401617407798767], all client accs: [0.7651332020759583, 0.7858964800834656],  alphas:tensor([0.7467, 0.0000, 0.0000, 0.2533], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:01,295 - utils - INFO - stage2_gradient_single_runtime: 0.006276369094848633
2023-09-28 23:29:01,300 - utils - INFO - 1, epoch: 1008, all client loss: [0.5316819548606873, 0.47149088978767395], all pred client disparities: [0.010259032249450684, 0.003810778260231018], all client disparities: [0.015942007303237915, 0.008401617407798767], all client accs: [0.7651332020759583, 0.7862386703491211],  alphas:tensor([0.4604, 0.0000, 0.0996, 0.4400], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:01,538 - utils - INFO - stage2_gradient_single_runtime: 0.006231069564819336
2023-09-28 23:29:01,543 - utils - INFO - 1, epoch: 1009, all client loss: [0.531715989112854, 0.47158461809158325], all pred client disparities: [0.010176926851272583, 0.0037306994199752808], all client disparities: [0.015942007303237915, 0.0072600096464157104], all client accs: [0.7651332020759583, 0.7768445611000061],  alphas:tensor([0.4601, 0.0000, 0.0999, 0.4400], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:01,779 - utils - INFO - stage2_gradient_single_runtime: 0.006264448165893555
2023-09-28 23:29:01,784 - utils - INFO - 1, epoch: 1010, all client loss: [0.5317505598068237, 0.47167810797691345], all pred client disparities: [0.010094016790390015, 0.0036503374576568604], all client disparities: [0.015942007303237915, 0.0029372423887252808], all client accs: [0.7651332020759583, 0.7766579389572144],  alphas:tensor([0.4598, 0.0000, 0.1002, 0.4400], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:02,018 - utils - INFO - stage2_gradient_single_runtime: 0.006221294403076172
2023-09-28 23:29:02,023 - utils - INFO - 1, epoch: 1011, all client loss: [0.5317858457565308, 0.47177135944366455], all pred client disparities: [0.010010242462158203, 0.0035696178674697876], all client disparities: [0.015942007303237915, 0.0029372423887252808], all client accs: [0.7651332020759583, 0.7766579389572144],  alphas:tensor([0.4596, 0.0000, 0.1005, 0.4400], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:02,263 - utils - INFO - stage2_gradient_single_runtime: 0.006224155426025391
2023-09-28 23:29:02,268 - utils - INFO - 1, epoch: 1012, all client loss: [0.5318216681480408, 0.4718644320964813], all pred client disparities: [0.00992557406425476, 0.0034886151552200317], all client disparities: [0.015942007303237915, 0.0029372423887252808], all client accs: [0.7651332020759583, 0.7766579389572144],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:02,508 - utils - INFO - stage2_gradient_single_runtime: 0.0062770843505859375
2023-09-28 23:29:02,514 - utils - INFO - 1, epoch: 1013, all client loss: [0.5316824316978455, 0.47161075472831726], all pred client disparities: [0.010112375020980835, 0.003563418984413147], all client disparities: [0.015942007303237915, 0.005098626017570496], all client accs: [0.7651332020759583, 0.7767512798309326],  alphas:tensor([0.4600, 0.0000, 0.1003, 0.4397], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:02,748 - utils - INFO - stage2_gradient_single_runtime: 0.006277799606323242
2023-09-28 23:29:02,753 - utils - INFO - 1, epoch: 1014, all client loss: [0.5317164063453674, 0.47170454263687134], all pred client disparities: [0.01003006100654602, 0.003484070301055908], all client disparities: [0.015942007303237915, 0.0029372423887252808], all client accs: [0.7651332020759583, 0.7766579389572144],  alphas:tensor([0.4597, 0.0000, 0.1006, 0.4397], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:02,988 - utils - INFO - stage2_gradient_single_runtime: 0.006251811981201172
2023-09-28 23:29:02,993 - utils - INFO - 1, epoch: 1015, all client loss: [0.5317510962486267, 0.4717981219291687], all pred client disparities: [0.009946942329406738, 0.0034044384956359863], all client disparities: [0.015942007303237915, 0.0029372423887252808], all client accs: [0.7651332020759583, 0.7766579389572144],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:03,227 - utils - INFO - stage2_gradient_single_runtime: 0.006201028823852539
2023-09-28 23:29:03,232 - utils - INFO - 1, epoch: 1016, all client loss: [0.5316126942634583, 0.47154438495635986], all pred client disparities: [0.010131925344467163, 0.003478512167930603], all client disparities: [0.015942007303237915, 0.0072600096464157104], all client accs: [0.7651332020759583, 0.7768445611000061],  alphas:tensor([0.4602, 0.0000, 0.1004, 0.4394], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:03,468 - utils - INFO - stage2_gradient_single_runtime: 0.006261348724365234
2023-09-28 23:29:03,473 - utils - INFO - 1, epoch: 1017, all client loss: [0.5316455960273743, 0.47163867950439453], all pred client disparities: [0.010051071643829346, 0.0034005343914031982], all client disparities: [0.015942007303237915, 0.005098626017570496], all client accs: [0.7651332020759583, 0.7767512798309326],  alphas:tensor([0.4599, 0.0000, 0.1007, 0.4394], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:03,709 - utils - INFO - stage2_gradient_single_runtime: 0.006205320358276367
2023-09-28 23:29:03,713 - utils - INFO - 1, epoch: 1018, all client loss: [0.5316789746284485, 0.47173282504081726], all pred client disparities: [0.009969443082809448, 0.003322288393974304], all client disparities: [0.015942007303237915, 0.0029372423887252808], all client accs: [0.7651332020759583, 0.7766579389572144],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:03,951 - utils - INFO - stage2_gradient_single_runtime: 0.0062694549560546875
2023-09-28 23:29:03,956 - utils - INFO - 1, epoch: 1019, all client loss: [0.5315415859222412, 0.4714789390563965], all pred client disparities: [0.010152608156204224, 0.003395557403564453], all client disparities: [0.015942007303237915, 0.007333084940910339], all client accs: [0.7651332020759583, 0.7768756747245789],  alphas:tensor([0.4604, 0.0000, 0.1005, 0.4391], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:04,038 - utils - INFO - valid: True, epoch: 1019, loss: [0.5808334946632385, 0.4712381064891815], accuracy: [0.7348066568374634, 0.7788819670677185], mean_accuracy:0.7568443119525909,variance_accuracy:0.022037655115127563, disparity: [0.019696980714797974, 0.01492348313331604], mean_disparity:0.017310231924057007,variance_disparity:0.002386748790740967, pred_disparity: [0.012749284505844116, 0.0005023926496505737]
2023-09-28 23:29:04,173 - utils - INFO - global_valid: True, epoch: 1019,  global_loss: 0.47245657444000244, global_accuracy: 0.8101729931682189,  global_disparity:0.01699639856815338, global_pred_disparity: 0.0032395273447036743,
2023-09-28 23:29:04,409 - utils - INFO - stage2_gradient_single_runtime: 0.006269216537475586
2023-09-28 23:29:04,415 - utils - INFO - 1, epoch: 1020, all client loss: [0.5315732359886169, 0.4715737998485565], all pred client disparities: [0.010073184967041016, 0.00331898033618927], all client disparities: [0.015942007303237915, 0.007333084940910339], all client accs: [0.7651332020759583, 0.7768756747245789],  alphas:tensor([0.7430, 0.0000, 0.0000, 0.2570], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:04,650 - utils - INFO - stage2_gradient_single_runtime: 0.006219387054443359
2023-09-28 23:29:04,655 - utils - INFO - 1, epoch: 1021, all client loss: [0.5316235423088074, 0.4712999761104584], all pred client disparities: [0.01009783148765564, 0.003128156065940857], all client disparities: [0.015942007303237915, 0.005829513072967529], all client accs: [0.7651332020759583, 0.7770623564720154],  alphas:tensor([0.4598, 0.0000, 0.1007, 0.4395], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:04,890 - utils - INFO - stage2_gradient_single_runtime: 0.006310224533081055
2023-09-28 23:29:04,895 - utils - INFO - 1, epoch: 1022, all client loss: [0.5316551923751831, 0.47139492630958557], all pred client disparities: [0.01001766324043274, 0.00305233895778656], all client disparities: [0.015942007303237915, 0.0036681294441223145], all client accs: [0.7651332020759583, 0.7769690155982971],  alphas:tensor([0.4595, 0.0000, 0.1010, 0.4395], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:05,132 - utils - INFO - stage2_gradient_single_runtime: 0.006184577941894531
2023-09-28 23:29:05,137 - utils - INFO - 1, epoch: 1023, all client loss: [0.5316873788833618, 0.47148972749710083], all pred client disparities: [0.009936720132827759, 0.00297622405923903], all client disparities: [0.015942007303237915, 0.0036681294441223145], all client accs: [0.7651332020759583, 0.7769690155982971],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:05,373 - utils - INFO - stage2_gradient_single_runtime: 0.006217002868652344
2023-09-28 23:29:05,378 - utils - INFO - 1, epoch: 1024, all client loss: [0.5312291979789734, 0.47207197546958923], all pred client disparities: [0.010128289461135864, 0.0037049949169158936], all client disparities: [0.015942007303237915, 0.008620887994766235], all client accs: [0.7651332020759583, 0.7858031392097473],  alphas:tensor([0.7603, 0.0000, 0.0000, 0.2397], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:05,614 - utils - INFO - stage2_gradient_single_runtime: 0.006238460540771484
2023-09-28 23:29:05,619 - utils - INFO - 1, epoch: 1025, all client loss: [0.5312707424163818, 0.47178953886032104], all pred client disparities: [0.010161787271499634, 0.0035193562507629395], all client disparities: [0.015942007303237915, 0.008255437016487122], all client accs: [0.7651332020759583, 0.7862075567245483],  alphas:tensor([0.7548, 0.0000, 0.0000, 0.2452], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:05,853 - utils - INFO - stage2_gradient_single_runtime: 0.006317138671875
2023-09-28 23:29:05,858 - utils - INFO - 1, epoch: 1026, all client loss: [0.5313141345977783, 0.4715110659599304], all pred client disparities: [0.010193824768066406, 0.003333047032356262], all client disparities: [0.015942007303237915, 0.0072600096464157104], all client accs: [0.7651332020759583, 0.7769067883491516],  alphas:tensor([0.7494, 0.0000, 0.0000, 0.2506], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:06,094 - utils - INFO - stage2_gradient_single_runtime: 0.006247520446777344
2023-09-28 23:29:06,099 - utils - INFO - 1, epoch: 1027, all client loss: [0.5313596129417419, 0.4712364971637726], all pred client disparities: [0.010224193334579468, 0.0031462162733078003], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7771245241165161],  alphas:tensor([0.4609, 0.0000, 0.1007, 0.4383], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:06,333 - utils - INFO - stage2_gradient_single_runtime: 0.006235837936401367
2023-09-28 23:29:06,338 - utils - INFO - 1, epoch: 1028, all client loss: [0.5313879251480103, 0.471332848072052], all pred client disparities: [0.01014852523803711, 0.0030735433101654053], all client disparities: [0.015942007303237915, 0.007844716310501099], all client accs: [0.7651332020759583, 0.7771556377410889],  alphas:tensor([0.4607, 0.0000, 0.1010, 0.4383], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:06,571 - utils - INFO - stage2_gradient_single_runtime: 0.006219387054443359
2023-09-28 23:29:06,576 - utils - INFO - 1, epoch: 1029, all client loss: [0.5314167141914368, 0.4714290201663971], all pred client disparities: [0.01007223129272461, 0.003000602126121521], all client disparities: [0.015942007303237915, 0.007844716310501099], all client accs: [0.7651332020759583, 0.7771556377410889],  alphas:tensor([0.7442, 0.0000, 0.0000, 0.2558], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:06,812 - utils - INFO - stage2_gradient_single_runtime: 0.006241798400878906
2023-09-28 23:29:06,816 - utils - INFO - 1, epoch: 1030, all client loss: [0.5314642190933228, 0.4711570739746094], all pred client disparities: [0.01010042428970337, 0.0028133243322372437], all client disparities: [0.015942007303237915, 0.0057564228773117065], all client accs: [0.7651332020759583, 0.7773422598838806],  alphas:tensor([0.4601, 0.0000, 0.1012, 0.4387], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:07,050 - utils - INFO - stage2_gradient_single_runtime: 0.006223440170288086
2023-09-28 23:29:07,055 - utils - INFO - 1, epoch: 1031, all client loss: [0.5314928889274597, 0.4712534248828888], all pred client disparities: [0.01002359390258789, 0.0027412772178649902], all client disparities: [0.015942007303237915, 0.0057564228773117065], all client accs: [0.7651332020759583, 0.7773422598838806],  alphas:tensor([0.4599, 0.0000, 0.1014, 0.4387], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:07,291 - utils - INFO - stage2_gradient_single_runtime: 0.006223440170288086
2023-09-28 23:29:07,296 - utils - INFO - 1, epoch: 1032, all client loss: [0.5315220355987549, 0.47134965658187866], all pred client disparities: [0.00994613766670227, 0.0026689767837524414], all client disparities: [0.015942007303237915, 0.005902588367462158], all client accs: [0.7651332020759583, 0.7773112058639526],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:07,530 - utils - INFO - stage2_gradient_single_runtime: 0.006315708160400391
2023-09-28 23:29:07,535 - utils - INFO - 1, epoch: 1033, all client loss: [0.5310710072517395, 0.47192704677581787], all pred client disparities: [0.01012393832206726, 0.0033911466598510742], all client disparities: [0.015942007303237915, 0.008401617407798767], all client accs: [0.7651332020759583, 0.7860831022262573],  alphas:tensor([0.7619, 0.0000, 0.0000, 0.2381], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:07,769 - utils - INFO - stage2_gradient_single_runtime: 0.006223440170288086
2023-09-28 23:29:07,774 - utils - INFO - 1, epoch: 1034, all client loss: [0.5311098694801331, 0.471646249294281], all pred client disparities: [0.010160565376281738, 0.003208845853805542], all client disparities: [0.015942007303237915, 0.007670730352401733], all client accs: [0.7651332020759583, 0.7865496873855591],  alphas:tensor([0.7564, 0.0000, 0.0000, 0.2436], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:08,010 - utils - INFO - stage2_gradient_single_runtime: 0.0062525272369384766
2023-09-28 23:29:08,015 - utils - INFO - 1, epoch: 1035, all client loss: [0.5311506390571594, 0.4713693857192993], all pred client disparities: [0.010195761919021606, 0.0030259937047958374], all client disparities: [0.015942007303237915, 0.007844716310501099], all client accs: [0.7651332020759583, 0.7771556377410889],  alphas:tensor([0.7511, 0.0000, 0.0000, 0.2489], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:08,258 - utils - INFO - stage2_gradient_single_runtime: 0.006845712661743164
2023-09-28 23:29:08,263 - utils - INFO - 1, epoch: 1036, all client loss: [0.531193196773529, 0.4710964560508728], all pred client disparities: [0.010229438543319702, 0.002842649817466736], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7771245241165161],  alphas:tensor([0.4613, 0.0000, 0.1012, 0.4375], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:08,503 - utils - INFO - stage2_gradient_single_runtime: 0.006635904312133789
2023-09-28 23:29:08,508 - utils - INFO - 1, epoch: 1037, all client loss: [0.5312185883522034, 0.47119414806365967], all pred client disparities: [0.010156989097595215, 0.002773657441139221], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7771245241165161],  alphas:tensor([0.4611, 0.0000, 0.1014, 0.4375], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:08,759 - utils - INFO - stage2_gradient_single_runtime: 0.00644230842590332
2023-09-28 23:29:08,764 - utils - INFO - 1, epoch: 1038, all client loss: [0.5312443971633911, 0.471291720867157], all pred client disparities: [0.010083973407745361, 0.002704441547393799], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7771245241165161],  alphas:tensor([0.7461, 0.0000, 0.0000, 0.2539], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:09,003 - utils - INFO - stage2_gradient_single_runtime: 0.006271839141845703
2023-09-28 23:29:09,008 - utils - INFO - 1, epoch: 1039, all client loss: [0.5312888026237488, 0.47102129459381104], all pred client disparities: [0.01011580228805542, 0.002520844340324402], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([0.4606, 0.0000, 0.1016, 0.4378], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:09,093 - utils - INFO - valid: True, epoch: 1039, loss: [0.5808513164520264, 0.4707699418067932], accuracy: [0.7348066568374634, 0.7790062427520752], mean_accuracy:0.7569064497947693,variance_accuracy:0.022099792957305908, disparity: [0.019696980714797974, 0.015663117170333862], mean_disparity:0.017680048942565918,variance_disparity:0.0020169317722320557, pred_disparity: [0.01178017258644104, 0.001228928565979004]
2023-09-28 23:29:09,221 - utils - INFO - global_valid: True, epoch: 1039,  global_loss: 0.4719937741756439, global_accuracy: 0.8101544254773809,  global_disparity:0.017712727189064026, global_pred_disparity: 0.0039957016706466675,
2023-09-28 23:29:09,459 - utils - INFO - stage2_gradient_single_runtime: 0.006248950958251953
2023-09-28 23:29:09,464 - utils - INFO - 1, epoch: 1040, all client loss: [0.5313144326210022, 0.47111913561820984], all pred client disparities: [0.010042458772659302, 0.0024525970220565796], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7774356007575989],  alphas:tensor([0.4604, 0.0000, 0.1018, 0.4378], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:09,704 - utils - INFO - stage2_gradient_single_runtime: 0.006333351135253906
2023-09-28 23:29:09,709 - utils - INFO - 1, epoch: 1041, all client loss: [0.5313404202461243, 0.4712167978286743], all pred client disparities: [0.009968429803848267, 0.0023841410875320435], all client disparities: [0.015942007303237915, 0.005902588367462158], all client accs: [0.7651332020759583, 0.7773422598838806],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:09,903 - utils - INFO - stage2_gradient_single_runtime: 0.006238698959350586
2023-09-28 23:29:09,908 - utils - INFO - 1, epoch: 1042, all client loss: [0.5308969020843506, 0.4717896580696106], all pred client disparities: [0.010131776332855225, 0.003100275993347168], all client disparities: [0.015942007303237915, 0.008182346820831299], all client accs: [0.7651332020759583, 0.7865496873855591],  alphas:tensor([0.7642, 0.0000, 0.0000, 0.2358], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:10,153 - utils - INFO - stage2_gradient_single_runtime: 0.006314277648925781
2023-09-28 23:29:10,158 - utils - INFO - 1, epoch: 1043, all client loss: [0.5309327840805054, 0.471510112285614], all pred client disparities: [0.010171741247177124, 0.0029215365648269653], all client disparities: [0.015942007303237915, 0.007771626114845276], all client accs: [0.7651332020759583, 0.7771556377410889],  alphas:tensor([0.7588, 0.0000, 0.0000, 0.2412], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:10,422 - utils - INFO - stage2_gradient_single_runtime: 0.006264209747314453
2023-09-28 23:29:10,427 - utils - INFO - 1, epoch: 1044, all client loss: [0.5309705138206482, 0.4712345600128174], all pred client disparities: [0.010210394859313965, 0.0027423053979873657], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7771556377410889],  alphas:tensor([0.7534, 0.0000, 0.0000, 0.2466], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:10,696 - utils - INFO - stage2_gradient_single_runtime: 0.006346940994262695
2023-09-28 23:29:10,702 - utils - INFO - 1, epoch: 1045, all client loss: [0.5310099720954895, 0.4709629714488983], all pred client disparities: [0.01024770736694336, 0.002562612295150757], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7773112058639526],  alphas:tensor([0.4619, 0.0000, 0.1016, 0.4365], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:10,988 - utils - INFO - stage2_gradient_single_runtime: 0.006247043609619141
2023-09-28 23:29:10,993 - utils - INFO - 1, epoch: 1046, all client loss: [0.5310323238372803, 0.4710620641708374], all pred client disparities: [0.01017838716506958, 0.002497375011444092], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7772489786148071],  alphas:tensor([0.7485, 0.0000, 0.0000, 0.2515], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:11,275 - utils - INFO - stage2_gradient_single_runtime: 0.006336212158203125
2023-09-28 23:29:11,282 - utils - INFO - 1, epoch: 1047, all client loss: [0.5310733914375305, 0.47079354524612427], all pred client disparities: [0.010214298963546753, 0.0023175179958343506], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7775600552558899],  alphas:tensor([0.4614, 0.0000, 0.1017, 0.4369], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:11,530 - utils - INFO - stage2_gradient_single_runtime: 0.006287574768066406
2023-09-28 23:29:11,537 - utils - INFO - 1, epoch: 1048, all client loss: [0.5310953855514526, 0.4708929657936096], all pred client disparities: [0.010144710540771484, 0.002253308892250061], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7775600552558899],  alphas:tensor([0.4612, 0.0000, 0.1020, 0.4368], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:11,773 - utils - INFO - stage2_gradient_single_runtime: 0.006354331970214844
2023-09-28 23:29:11,780 - utils - INFO - 1, epoch: 1049, all client loss: [0.5311177372932434, 0.47099223732948303], all pred client disparities: [0.010074794292449951, 0.002188935875892639], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([0.4610, 0.0000, 0.1022, 0.4368], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:12,025 - utils - INFO - stage2_gradient_single_runtime: 0.006796121597290039
2023-09-28 23:29:12,032 - utils - INFO - 1, epoch: 1050, all client loss: [0.5311405062675476, 0.4710913896560669], all pred client disparities: [0.010004401206970215, 0.0021243542432785034], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([0.4608, 0.0000, 0.1024, 0.4367], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:12,267 - utils - INFO - stage2_gradient_single_runtime: 0.006273984909057617
2023-09-28 23:29:12,273 - utils - INFO - 1, epoch: 1051, all client loss: [0.5311636924743652, 0.4711904227733612], all pred client disparities: [0.009933650493621826, 0.00205954909324646], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:12,526 - utils - INFO - stage2_gradient_single_runtime: 0.006484270095825195
2023-09-28 23:29:12,531 - utils - INFO - 1, epoch: 1052, all client loss: [0.5310319066047668, 0.47093650698661804], all pred client disparities: [0.010104864835739136, 0.0021280795335769653], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7775600552558899],  alphas:tensor([0.4614, 0.0000, 0.1023, 0.4364], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:12,765 - utils - INFO - stage2_gradient_single_runtime: 0.0062274932861328125
2023-09-28 23:29:12,770 - utils - INFO - 1, epoch: 1053, all client loss: [0.5310534834861755, 0.4710361659526825], all pred client disparities: [0.010035783052444458, 0.0020648539066314697], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([0.4612, 0.0000, 0.1025, 0.4363], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:13,003 - utils - INFO - stage2_gradient_single_runtime: 0.006227254867553711
2023-09-28 23:29:13,010 - utils - INFO - 1, epoch: 1054, all client loss: [0.5310754179954529, 0.4711357057094574], all pred client disparities: [0.009966492652893066, 0.002001434564590454], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:13,274 - utils - INFO - stage2_gradient_single_runtime: 0.006588935852050781
2023-09-28 23:29:13,279 - utils - INFO - 1, epoch: 1055, all client loss: [0.5309450030326843, 0.4708813428878784], all pred client disparities: [0.010135352611541748, 0.0020687133073806763], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7775600552558899],  alphas:tensor([0.4617, 0.0000, 0.1023, 0.4359], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:13,594 - utils - INFO - stage2_gradient_single_runtime: 0.006325483322143555
2023-09-28 23:29:13,599 - utils - INFO - 1, epoch: 1056, all client loss: [0.5309653282165527, 0.47098150849342346], all pred client disparities: [0.010067611932754517, 0.002006828784942627], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([0.7474, 0.0000, 0.0000, 0.2526], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:13,827 - utils - INFO - stage2_gradient_single_runtime: 0.00628352165222168
2023-09-28 23:29:13,833 - utils - INFO - 1, epoch: 1057, all client loss: [0.5310049057006836, 0.470715194940567], all pred client disparities: [0.010105311870574951, 0.0018299520015716553], all client disparities: [0.015942007303237915, 0.008137062191963196], all client accs: [0.7651332020759583, 0.7775600552558899],  alphas:tensor([0.4613, 0.0000, 0.1025, 0.4362], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:14,062 - utils - INFO - stage2_gradient_single_runtime: 0.0062634944915771484
2023-09-28 23:29:14,067 - utils - INFO - 1, epoch: 1058, all client loss: [0.5310247540473938, 0.4708157479763031], all pred client disparities: [0.010037481784820557, 0.00176924467086792], all client disparities: [0.015942007303237915, 0.008137062191963196], all client accs: [0.7651332020759583, 0.7775600552558899],  alphas:tensor([0.4611, 0.0000, 0.1027, 0.4362], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:14,297 - utils - INFO - stage2_gradient_single_runtime: 0.006220340728759766
2023-09-28 23:29:14,302 - utils - INFO - 1, epoch: 1059, all client loss: [0.5310449004173279, 0.47091618180274963], all pred client disparities: [0.00996929407119751, 0.0017083436250686646], all client disparities: [0.015942007303237915, 0.008137062191963196], all client accs: [0.7651332020759583, 0.7775600552558899],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:14,428 - utils - INFO - valid: True, epoch: 1059, loss: [0.580891489982605, 0.47114795446395874], accuracy: [0.7348066568374634, 0.7783851027488708], mean_accuracy:0.7565958797931671,variance_accuracy:0.021789222955703735, disparity: [0.019696980714797974, 0.016981467604637146], mean_disparity:0.01833922415971756,variance_disparity:0.0013577565550804138, pred_disparity: [0.010485917329788208, 0.0011543631553649902]
2023-09-28 23:29:14,506 - utils - INFO - global_valid: True, epoch: 1059,  global_loss: 0.4723680019378662, global_accuracy: 0.8101905362905308,  global_disparity:0.01897396147251129, global_pred_disparity: 0.003974601626396179,
2023-09-28 23:29:14,739 - utils - INFO - stage2_gradient_single_runtime: 0.0062410831451416016
2023-09-28 23:29:14,743 - utils - INFO - 1, epoch: 1060, all client loss: [0.5306148529052734, 0.4714788794517517], all pred client disparities: [0.010107308626174927, 0.002410203218460083], all client disparities: [0.015942007303237915, 0.007333084940910339], all client accs: [0.7651332020759583, 0.7771245241165161],  alphas:tensor([0.7662, 0.0000, 0.0000, 0.2338], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:15,035 - utils - INFO - stage2_gradient_single_runtime: 0.006270170211791992
2023-09-28 23:29:15,040 - utils - INFO - 1, epoch: 1061, all client loss: [0.5306461453437805, 0.4712029695510864], all pred client disparities: [0.010152369737625122, 0.002237766981124878], all client disparities: [0.015942007303237915, 0.007552355527877808], all client accs: [0.7651332020759583, 0.7772489786148071],  alphas:tensor([0.7609, 0.0000, 0.0000, 0.2391], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:15,268 - utils - INFO - stage2_gradient_single_runtime: 0.006233930587768555
2023-09-28 23:29:15,273 - utils - INFO - 1, epoch: 1062, all client loss: [0.5306791067123413, 0.4709310531616211], all pred client disparities: [0.010196119546890259, 0.002065032720565796], all client disparities: [0.015942007303237915, 0.007844716310501099], all client accs: [0.7651332020759583, 0.7774044871330261],  alphas:tensor([0.7556, 0.0000, 0.0000, 0.2444], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:15,510 - utils - INFO - stage2_gradient_single_runtime: 0.0062754154205322266
2023-09-28 23:29:15,515 - utils - INFO - 1, epoch: 1063, all client loss: [0.5307137370109558, 0.47066301107406616], all pred client disparities: [0.010238885879516602, 0.0018919259309768677], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7776533365249634],  alphas:tensor([0.4627, 0.0000, 0.1024, 0.4348], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:15,746 - utils - INFO - stage2_gradient_single_runtime: 0.006232261657714844
2023-09-28 23:29:15,751 - utils - INFO - 1, epoch: 1064, all client loss: [0.5307306051254272, 0.47076472640037537], all pred client disparities: [0.01017463207244873, 0.0018339604139328003], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7776222229003906],  alphas:tensor([0.7510, 0.0000, 0.0000, 0.2490], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:15,982 - utils - INFO - stage2_gradient_single_runtime: 0.006278276443481445
2023-09-28 23:29:15,987 - utils - INFO - 1, epoch: 1065, all client loss: [0.5307665467262268, 0.47049978375434875], all pred client disparities: [0.010216236114501953, 0.0016608983278274536], all client disparities: [0.015942007303237915, 0.007990896701812744], all client accs: [0.7651332020759583, 0.7777777910232544],  alphas:tensor([0.4623, 0.0000, 0.1025, 0.4351], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:16,219 - utils - INFO - stage2_gradient_single_runtime: 0.006255388259887695
2023-09-28 23:29:16,224 - utils - INFO - 1, epoch: 1066, all client loss: [0.5307828187942505, 0.4706019163131714], all pred client disparities: [0.010152041912078857, 0.0016041398048400879], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7776844501495361],  alphas:tensor([0.4622, 0.0000, 0.1028, 0.4350], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:16,499 - utils - INFO - stage2_gradient_single_runtime: 0.006263017654418945
2023-09-28 23:29:16,504 - utils - INFO - 1, epoch: 1067, all client loss: [0.5307994484901428, 0.4707038998603821], all pred client disparities: [0.01008760929107666, 0.0015472173690795898], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7775911092758179],  alphas:tensor([0.4620, 0.0000, 0.1030, 0.4350], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:16,737 - utils - INFO - stage2_gradient_single_runtime: 0.0063323974609375
2023-09-28 23:29:16,743 - utils - INFO - 1, epoch: 1068, all client loss: [0.5308164358139038, 0.4708057940006256], all pred client disparities: [0.010022848844528198, 0.0014901161193847656], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7775911092758179],  alphas:tensor([0.4619, 0.0000, 0.1032, 0.4349], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:16,978 - utils - INFO - stage2_gradient_single_runtime: 0.006126880645751953
2023-09-28 23:29:16,983 - utils - INFO - 1, epoch: 1069, all client loss: [0.5308337211608887, 0.4709075689315796], all pred client disparities: [0.009957879781723022, 0.0014328211545944214], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7775600552558899],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:17,219 - utils - INFO - stage2_gradient_single_runtime: 0.0062329769134521484
2023-09-28 23:29:17,224 - utils - INFO - 1, epoch: 1070, all client loss: [0.5307062268257141, 0.4706527888774872], all pred client disparities: [0.010120958089828491, 0.001497507095336914], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7776533365249634],  alphas:tensor([0.4625, 0.0000, 0.1030, 0.4345], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:17,458 - utils - INFO - stage2_gradient_single_runtime: 0.0063152313232421875
2023-09-28 23:29:17,463 - utils - INFO - 1, epoch: 1071, all client loss: [0.5307220220565796, 0.4707551598548889], all pred client disparities: [0.010057508945465088, 0.0014417022466659546], all client disparities: [0.015942007303237915, 0.007917806506156921], all client accs: [0.7651332020759583, 0.7776533365249634],  alphas:tensor([0.7496, 0.0000, 0.0000, 0.2504], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:17,698 - utils - INFO - stage2_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:29:17,703 - utils - INFO - 1, epoch: 1072, all client loss: [0.5307571291923523, 0.4704916179180145], all pred client disparities: [0.010099798440933228, 0.0012706220149993896], all client disparities: [0.015942007303237915, 0.007270440459251404], all client accs: [0.7651332020759583, 0.7778710722923279],  alphas:tensor([0.4621, 0.0000, 0.1031, 0.4348], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:17,991 - utils - INFO - stage2_gradient_single_runtime: 0.006241321563720703
2023-09-28 23:29:17,998 - utils - INFO - 1, epoch: 1073, all client loss: [0.5307722687721252, 0.4705944359302521], all pred client disparities: [0.010036319494247437, 0.0012161284685134888], all client disparities: [0.015942007303237915, 0.007343515753746033], all client accs: [0.7651332020759583, 0.7776844501495361],  alphas:tensor([0.4620, 0.0000, 0.1034, 0.4347], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:18,228 - utils - INFO - stage2_gradient_single_runtime: 0.006299495697021484
2023-09-28 23:29:18,234 - utils - INFO - 1, epoch: 1074, all client loss: [0.5307877063751221, 0.4706971049308777], all pred client disparities: [0.009972691535949707, 0.0011614412069320679], all client disparities: [0.015942007303237915, 0.008063971996307373], all client accs: [0.7651332020759583, 0.7775911092758179],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:18,519 - utils - INFO - stage2_gradient_single_runtime: 0.006617307662963867
2023-09-28 23:29:18,525 - utils - INFO - 1, epoch: 1075, all client loss: [0.530369222164154, 0.4712519645690918], all pred client disparities: [0.010089695453643799, 0.001852080225944519], all client disparities: [0.015942007303237915, 0.007406175136566162], all client accs: [0.7651332020759583, 0.7771556377410889],  alphas:tensor([0.7690, 0.0000, 0.0000, 0.2310], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:18,798 - utils - INFO - stage2_gradient_single_runtime: 0.0062601566314697266
2023-09-28 23:29:18,802 - utils - INFO - 1, epoch: 1076, all client loss: [0.5303964018821716, 0.47097840905189514], all pred client disparities: [0.010138720273971558, 0.0016851425170898438], all client disparities: [0.015942007303237915, 0.00762544572353363], all client accs: [0.7651332020759583, 0.7774044871330261],  alphas:tensor([0.7637, 0.0000, 0.0000, 0.2363], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:19,043 - utils - INFO - stage2_gradient_single_runtime: 0.006285667419433594
2023-09-28 23:29:19,049 - utils - INFO - 1, epoch: 1077, all client loss: [0.5304251909255981, 0.47070884704589844], all pred client disparities: [0.010186821222305298, 0.0015179216861724854], all client disparities: [0.015942007303237915, 0.00762544572353363], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([0.7584, 0.0000, 0.0000, 0.2416], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:19,328 - utils - INFO - stage2_gradient_single_runtime: 0.00632786750793457
2023-09-28 23:29:19,333 - utils - INFO - 1, epoch: 1078, all client loss: [0.5304552912712097, 0.4704432189464569], all pred client disparities: [0.010233938694000244, 0.0013504624366760254], all client disparities: [0.015942007303237915, 0.007270440459251404], all client accs: [0.7651332020759583, 0.7778399586677551],  alphas:tensor([0.4636, 0.0000, 0.1031, 0.4333], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:19,565 - utils - INFO - stage2_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:29:19,570 - utils - INFO - 1, epoch: 1079, all client loss: [0.5304678082466125, 0.47054702043533325], all pred client disparities: [0.010173648595809937, 0.001298472285270691], all client disparities: [0.015942007303237915, 0.0070511698722839355], all client accs: [0.7651332020759583, 0.7777777910232544],  alphas:tensor([0.7541, 0.0000, 0.0000, 0.2459], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:19,652 - utils - INFO - valid: True, epoch: 1079, loss: [0.5807791948318481, 0.469911128282547], accuracy: [0.7348066568374634, 0.7790683507919312], mean_accuracy:0.7569375038146973,variance_accuracy:0.022130846977233887, disparity: [0.019696980714797974, 0.015811055898666382], mean_disparity:0.017754018306732178,variance_disparity:0.001942962408065796, pred_disparity: [0.009517043828964233, 0.0022393763065338135]
2023-09-28 23:29:19,782 - utils - INFO - global_valid: True, epoch: 1079,  global_loss: 0.4711437225341797, global_accuracy: 0.8103073580120528,  global_disparity:0.01785600185394287, global_pred_disparity: 0.005083516240119934,
2023-09-28 23:29:20,022 - utils - INFO - stage2_gradient_single_runtime: 0.0063397884368896484
2023-09-28 23:29:20,027 - utils - INFO - 1, epoch: 1080, all client loss: [0.5304990410804749, 0.4702843427658081], all pred client disparities: [0.01022002100944519, 0.0011311769485473633], all client disparities: [0.015942007303237915, 0.007343515753746033], all client accs: [0.7651332020759583, 0.7779021859169006],  alphas:tensor([0.4633, 0.0000, 0.1032, 0.4335], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:20,256 - utils - INFO - stage2_gradient_single_runtime: 0.006273746490478516
2023-09-28 23:29:20,261 - utils - INFO - 1, epoch: 1081, all client loss: [0.5305107831954956, 0.4703886806964874], all pred client disparities: [0.01015964150428772, 0.0010805577039718628], all client disparities: [0.015942007303237915, 0.007270440459251404], all client accs: [0.7651332020759583, 0.7778399586677551],  alphas:tensor([0.4632, 0.0000, 0.1034, 0.4334], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:20,496 - utils - INFO - stage2_gradient_single_runtime: 0.006303071975708008
2023-09-28 23:29:20,501 - utils - INFO - 1, epoch: 1082, all client loss: [0.5305227637290955, 0.4704928696155548], all pred client disparities: [0.010099291801452637, 0.0010297000408172607], all client disparities: [0.015942007303237915, 0.007197350263595581], all client accs: [0.7651332020759583, 0.7778399586677551],  alphas:tensor([0.4631, 0.0000, 0.1036, 0.4333], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:20,773 - utils - INFO - stage2_gradient_single_runtime: 0.006280183792114258
2023-09-28 23:29:20,778 - utils - INFO - 1, epoch: 1083, all client loss: [0.530535101890564, 0.47059696912765503], all pred client disparities: [0.01003885269165039, 0.0009786635637283325], all client disparities: [0.015942007303237915, 0.007124260067939758], all client accs: [0.7651332020759583, 0.7777155637741089],  alphas:tensor([0.7515, 0.0000, 0.0000, 0.2485], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:21,008 - utils - INFO - stage2_gradient_single_runtime: 0.006279706954956055
2023-09-28 23:29:21,013 - utils - INFO - 1, epoch: 1084, all client loss: [0.5305667519569397, 0.47033533453941345], all pred client disparities: [0.010084718465805054, 0.0008121877908706665], all client disparities: [0.015942007303237915, 0.007270440459251404], all client accs: [0.7651332020759583, 0.7779644131660461],  alphas:tensor([0.4628, 0.0000, 0.1037, 0.4335], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:21,243 - utils - INFO - stage2_gradient_single_runtime: 0.0062482357025146484
2023-09-28 23:29:21,248 - utils - INFO - 1, epoch: 1085, all client loss: [0.5305782556533813, 0.47043994069099426], all pred client disparities: [0.010024309158325195, 0.0007625371217727661], all client disparities: [0.015942007303237915, 0.007197350263595581], all client accs: [0.7651332020759583, 0.7779021859169006],  alphas:tensor([0.4627, 0.0000, 0.1039, 0.4334], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:21,480 - utils - INFO - stage2_gradient_single_runtime: 0.006218433380126953
2023-09-28 23:29:21,485 - utils - INFO - 1, epoch: 1086, all client loss: [0.5305900573730469, 0.47054436802864075], all pred client disparities: [0.009963750839233398, 0.0007127374410629272], all client disparities: [0.015942007303237915, 0.007124260067939758], all client accs: [0.7651332020759583, 0.7778089046478271],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:21,717 - utils - INFO - stage2_gradient_single_runtime: 0.006232738494873047
2023-09-28 23:29:21,722 - utils - INFO - 1, epoch: 1087, all client loss: [0.530180811882019, 0.47109320759773254], all pred client disparities: [0.010064840316772461, 0.0013944655656814575], all client disparities: [0.005072444677352905, 0.008199736475944519], all client accs: [0.7602905631065369, 0.7772489786148071],  alphas:tensor([0.7715, 0.0000, 0.0000, 0.2285], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:21,954 - utils - INFO - stage2_gradient_single_runtime: 0.006545543670654297
2023-09-28 23:29:21,959 - utils - INFO - 1, epoch: 1088, all client loss: [0.530204713344574, 0.47082120180130005], all pred client disparities: [0.01011696457862854, 0.001231849193572998], all client disparities: [0.015942007303237915, 0.00762544572353363], all client accs: [0.7651332020759583, 0.7774978280067444],  alphas:tensor([0.7662, 0.0000, 0.0000, 0.2338], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:22,192 - utils - INFO - stage2_gradient_single_runtime: 0.0062215328216552734
2023-09-28 23:29:22,197 - utils - INFO - 1, epoch: 1089, all client loss: [0.5302301049232483, 0.4705532491207123], all pred client disparities: [0.010168105363845825, 0.0010690242052078247], all client disparities: [0.015942007303237915, 0.00762544572353363], all client accs: [0.7651332020759583, 0.7778089046478271],  alphas:tensor([0.7610, 0.0000, 0.0000, 0.2390], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:22,429 - utils - INFO - stage2_gradient_single_runtime: 0.006276130676269531
2023-09-28 23:29:22,435 - utils - INFO - 1, epoch: 1090, all client loss: [0.5302568078041077, 0.4702892005443573], all pred client disparities: [0.010218411684036255, 0.0009060502052307129], all client disparities: [0.015942007303237915, 0.007270440459251404], all client accs: [0.7651332020759583, 0.7779021859169006],  alphas:tensor([0.7558, 0.0000, 0.0000, 0.2442], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:22,666 - utils - INFO - stage2_gradient_single_runtime: 0.0062825679779052734
2023-09-28 23:29:22,671 - utils - INFO - 1, epoch: 1091, all client loss: [0.5302849411964417, 0.4700290560722351], all pred client disparities: [0.010267883539199829, 0.0007430315017700195], all client disparities: [0.015942007303237915, 0.006048768758773804], all client accs: [0.7651332020759583, 0.7779955267906189],  alphas:tensor([0.4643, 0.0000, 0.1035, 0.4322], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:22,911 - utils - INFO - stage2_gradient_single_runtime: 0.006212472915649414
2023-09-28 23:29:22,917 - utils - INFO - 1, epoch: 1092, all client loss: [0.530292809009552, 0.4701351523399353], all pred client disparities: [0.010210484266281128, 0.0006972551345825195], all client disparities: [0.015942007303237915, 0.006048768758773804], all client accs: [0.7651332020759583, 0.7779955267906189],  alphas:tensor([0.4642, 0.0000, 0.1037, 0.4321], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:23,102 - utils - INFO - stage2_gradient_single_runtime: 0.00622105598449707
2023-09-28 23:29:23,108 - utils - INFO - 1, epoch: 1093, all client loss: [0.5303010940551758, 0.4702411890029907], all pred client disparities: [0.010153055191040039, 0.0006512850522994995], all client disparities: [0.015942007303237915, 0.007270440459251404], all client accs: [0.7651332020759583, 0.7779955267906189],  alphas:tensor([0.4641, 0.0000, 0.1039, 0.4320], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:23,340 - utils - INFO - stage2_gradient_single_runtime: 0.006233692169189453
2023-09-28 23:29:23,346 - utils - INFO - 1, epoch: 1094, all client loss: [0.5303096175193787, 0.4703470468521118], all pred client disparities: [0.010095566511154175, 0.000605165958404541], all client disparities: [0.015942007303237915, 0.007270440459251404], all client accs: [0.7651332020759583, 0.7779955267906189],  alphas:tensor([0.7539, 0.0000, 0.0000, 0.2461], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:23,578 - utils - INFO - stage2_gradient_single_runtime: 0.006345510482788086
2023-09-28 23:29:23,585 - utils - INFO - 1, epoch: 1095, all client loss: [0.5303376913070679, 0.4700877070426941], all pred client disparities: [0.010144859552383423, 0.0004430711269378662], all client disparities: [0.015942007303237915, 0.006194949150085449], all client accs: [0.7651332020759583, 0.7780266404151917],  alphas:tensor([0.4639, 0.0000, 0.1039, 0.4322], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:23,818 - utils - INFO - stage2_gradient_single_runtime: 0.006393909454345703
2023-09-28 23:29:23,825 - utils - INFO - 1, epoch: 1096, all client loss: [0.5303453207015991, 0.47019416093826294], all pred client disparities: [0.01008749008178711, 0.00039839744567871094], all client disparities: [0.015942007303237915, 0.006194949150085449], all client accs: [0.7651332020759583, 0.7780266404151917],  alphas:tensor([0.4638, 0.0000, 0.1041, 0.4321], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:24,066 - utils - INFO - stage2_gradient_single_runtime: 0.0063626766204833984
2023-09-28 23:29:24,072 - utils - INFO - 1, epoch: 1097, all client loss: [0.5303532481193542, 0.47030043601989746], all pred client disparities: [0.010030150413513184, 0.00035353004932403564], all client disparities: [0.015942007303237915, 0.006194949150085449], all client accs: [0.7651332020759583, 0.7780266404151917],  alphas:tensor([0.4637, 0.0000, 0.1043, 0.4320], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:24,312 - utils - INFO - stage2_gradient_single_runtime: 0.006340742111206055
2023-09-28 23:29:24,317 - utils - INFO - 1, epoch: 1098, all client loss: [0.5303614735603333, 0.47040659189224243], all pred client disparities: [0.009972631931304932, 0.0003084540367126465], all client disparities: [0.015942007303237915, 0.006194949150085449], all client accs: [0.7627118825912476, 0.7780266404151917],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:24,559 - utils - INFO - stage2_gradient_single_runtime: 0.006246328353881836
2023-09-28 23:29:24,567 - utils - INFO - 1, epoch: 1099, all client loss: [0.5302393436431885, 0.4701516628265381], all pred client disparities: [0.010124295949935913, 0.00036847591400146484], all client disparities: [0.015942007303237915, 0.006194949150085449], all client accs: [0.7651332020759583, 0.7780266404151917],  alphas:tensor([0.4644, 0.0000, 0.1041, 0.4315], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:24,699 - utils - INFO - valid: True, epoch: 1099, loss: [0.5809122323989868, 0.46988359093666077], accuracy: [0.7403315305709839, 0.7790683507919312], mean_accuracy:0.7596999406814575,variance_accuracy:0.019368410110473633, disparity: [0.024242430925369263, 0.016254842281341553], mean_disparity:0.020248636603355408,variance_disparity:0.003993794322013855, pred_disparity: [0.00844871997833252, 0.0029121190309524536]
2023-09-28 23:29:24,778 - utils - INFO - global_valid: True, epoch: 1099,  global_loss: 0.47111791372299194, global_accuracy: 0.8101587955757876,  global_disparity:0.018142521381378174, global_pred_disparity: 0.0057901740074157715,
2023-09-28 23:29:25,012 - utils - INFO - stage2_gradient_single_runtime: 0.006205558776855469
2023-09-28 23:29:25,017 - utils - INFO - 1, epoch: 1100, all client loss: [0.5302462577819824, 0.47025835514068604], all pred client disparities: [0.010067880153656006, 0.0003247261047363281], all client disparities: [0.015942007303237915, 0.006194949150085449], all client accs: [0.7627118825912476, 0.7780266404151917],  alphas:tensor([0.7538, 0.0000, 0.0000, 0.2462], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:25,254 - utils - INFO - stage2_gradient_single_runtime: 0.006306171417236328
2023-09-28 23:29:25,259 - utils - INFO - 1, epoch: 1101, all client loss: [0.5302730798721313, 0.4700004458427429], all pred client disparities: [0.010118305683135986, 0.0001647472381591797], all client disparities: [0.015942007303237915, 0.006268039345741272], all client accs: [0.7651332020759583, 0.7780888080596924],  alphas:tensor([0.4642, 0.0000, 0.1042, 0.4316], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:25,494 - utils - INFO - stage2_gradient_single_runtime: 0.006325244903564453
2023-09-28 23:29:25,499 - utils - INFO - 1, epoch: 1102, all client loss: [0.5302789807319641, 0.4701077342033386], all pred client disparities: [0.010061979293823242, 0.00012248754501342773], all client disparities: [0.015942007303237915, 0.006268039345741272], all client accs: [0.7627118825912476, 0.7780888080596924],  alphas:tensor([0.4641, 0.0000, 0.1044, 0.4315], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:25,733 - utils - INFO - stage2_gradient_single_runtime: 0.006328105926513672
2023-09-28 23:29:25,738 - utils - INFO - 1, epoch: 1103, all client loss: [0.5302852988243103, 0.47021484375], all pred client disparities: [0.010005772113800049, 8.00788402557373e-05], all client disparities: [0.015942007303237915, 0.006268039345741272], all client accs: [0.7627118825912476, 0.7780888080596924],  alphas:tensor([0.4640, 0.0000, 0.1046, 0.4314], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:25,990 - utils - INFO - stage2_gradient_single_runtime: 0.006483316421508789
2023-09-28 23:29:25,995 - utils - INFO - 1, epoch: 1104, all client loss: [0.5302918553352356, 0.4703218936920166], all pred client disparities: [0.009949415922164917, 3.738701707334258e-05], all client disparities: [0.005072444677352905, 0.006268039345741272], all client accs: [0.757869303226471, 0.7780888080596924],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:26,237 - utils - INFO - stage2_gradient_single_runtime: 0.006295204162597656
2023-09-28 23:29:26,242 - utils - INFO - 1, epoch: 1105, all client loss: [0.5301703810691833, 0.47006702423095703], all pred client disparities: [0.010099440813064575, 9.679794311523438e-05], all client disparities: [0.015942007303237915, 0.006988510489463806], all client accs: [0.7627118825912476, 0.7781199216842651],  alphas:tensor([0.4647, 0.0000, 0.1044, 0.4309], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:26,475 - utils - INFO - stage2_gradient_single_runtime: 0.006254911422729492
2023-09-28 23:29:26,480 - utils - INFO - 1, epoch: 1106, all client loss: [0.5301756858825684, 0.47017452120780945], all pred client disparities: [0.01004403829574585, 5.5447224440285936e-05], all client disparities: [0.005072444677352905, 0.006988510489463806], all client accs: [0.757869303226471, 0.7781199216842651],  alphas:tensor([0.4646, 0.0000, 0.1046, 0.4308], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:26,712 - utils - INFO - stage2_gradient_single_runtime: 0.006271839141845703
2023-09-28 23:29:26,717 - utils - INFO - 1, epoch: 1107, all client loss: [0.5301812887191772, 0.4702819287776947], all pred client disparities: [0.009988754987716675, 1.385807991027832e-05], all client disparities: [0.005072444677352905, 0.006988510489463806], all client accs: [0.757869303226471, 0.7781199216842651],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:26,949 - utils - INFO - stage2_gradient_single_runtime: 0.006272077560424805
2023-09-28 23:29:26,954 - utils - INFO - 1, epoch: 1108, all client loss: [0.5300617218017578, 0.47002604603767395], all pred client disparities: [0.010136187076568604, 7.136166095733643e-05], all client disparities: [0.005072444677352905, 0.007134690880775452], all client accs: [0.757869303226471, 0.7780266404151917],  alphas:tensor([0.4652, 0.0000, 0.1044, 0.4303], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:27,186 - utils - INFO - stage2_gradient_single_runtime: 0.006267070770263672
2023-09-28 23:29:27,191 - utils - INFO - 1, epoch: 1109, all client loss: [0.5300660729408264, 0.4701339602470398], all pred client disparities: [0.010081619024276733, 3.1039122404763475e-05], all client disparities: [0.005072444677352905, 0.007134690880775452], all client accs: [0.757869303226471, 0.7780266404151917],  alphas:tensor([0.5472, 0.0000, 0.2984, 0.1544], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:27,438 - utils - INFO - stage2_gradient_single_runtime: 0.0062408447265625
2023-09-28 23:29:27,443 - utils - INFO - 1, epoch: 1110, all client loss: [0.5299661755561829, 0.47020310163497925], all pred client disparities: [0.010018318891525269, 0.00031113624572753906], all client disparities: [0.005072444677352905, 0.006769239902496338], all client accs: [0.757869303226471, 0.7780266404151917],  alphas:tensor([0.7614, 0.0000, 0.0000, 0.2386], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:27,690 - utils - INFO - stage2_gradient_single_runtime: 0.006249189376831055
2023-09-28 23:29:27,694 - utils - INFO - 1, epoch: 1111, all client loss: [0.5299890637397766, 0.4699432849884033], all pred client disparities: [0.010072290897369385, 0.00015413761138916016], all client disparities: [0.005072444677352905, 0.007280871272087097], all client accs: [0.757869303226471, 0.7779955267906189],  alphas:tensor([0.4651, 0.0000, 0.1048, 0.4301], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:27,944 - utils - INFO - stage2_gradient_single_runtime: 0.007170915603637695
2023-09-28 23:29:27,949 - utils - INFO - 1, epoch: 1112, all client loss: [0.5299935340881348, 0.4700508713722229], all pred client disparities: [0.010017871856689453, 0.00011363625526428223], all client disparities: [0.005072444677352905, 0.007134690880775452], all client accs: [0.757869303226471, 0.7780266404151917],  alphas:tensor([0.7576, 0.0000, 0.0000, 0.2424], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:28,187 - utils - INFO - stage2_gradient_single_runtime: 0.007191896438598633
2023-09-28 23:29:28,191 - utils - INFO - 1, epoch: 1113, all client loss: [0.5300171971321106, 0.46979376673698425], all pred client disparities: [0.01007145643234253, 4.2945146560668945e-05], all client disparities: [0.005072444677352905, 0.007500126957893372], all client accs: [0.757869303226471, 0.7779332995414734],  alphas:tensor([7.6193e-17, 2.2074e-01, 1.9541e-01, 5.8386e-01], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:29:28,426 - utils - INFO - stage2_gradient_single_runtime: 0.0062563419342041016
2023-09-28 23:29:28,432 - utils - INFO - 1, epoch: 1114, all client loss: [0.5299500823020935, 0.4698854386806488], all pred client disparities: [0.009972870349884033, 0.00020354986190795898], all client disparities: [0.005072444677352905, 0.007280871272087097], all client accs: [0.757869303226471, 0.7779955267906189],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:28,678 - utils - INFO - stage2_gradient_single_runtime: 0.0071942806243896484
2023-09-28 23:29:28,682 - utils - INFO - 1, epoch: 1115, all client loss: [0.5295630097389221, 0.470420241355896], all pred client disparities: [0.010035306215286255, 0.0008709728717803955], all client disparities: [0.005072444677352905, 0.007698535919189453], all client accs: [0.757869303226471, 0.7775289416313171],  alphas:tensor([0.7798, 0.0000, 0.0000, 0.2202], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:28,922 - utils - INFO - stage2_gradient_single_runtime: 0.006280660629272461
2023-09-28 23:29:28,929 - utils - INFO - 1, epoch: 1116, all client loss: [0.5295798778533936, 0.4701518416404724], all pred client disparities: [0.010094225406646729, 0.000716671347618103], all client disparities: [0.005072444677352905, 0.006476894021034241], all client accs: [0.757869303226471, 0.7778399586677551],  alphas:tensor([0.7745, 0.0000, 0.0000, 0.2255], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:29,169 - utils - INFO - stage2_gradient_single_runtime: 0.006312131881713867
2023-09-28 23:29:29,175 - utils - INFO - 1, epoch: 1117, all client loss: [0.5295981168746948, 0.4698874354362488], all pred client disparities: [0.010152369737625122, 0.0005621165037155151], all client disparities: [0.005072444677352905, 0.006915420293807983], all client accs: [0.757869303226471, 0.7778089046478271],  alphas:tensor([0.7693, 0.0000, 0.0000, 0.2307], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:29,409 - utils - INFO - stage2_gradient_single_runtime: 0.006206512451171875
2023-09-28 23:29:29,415 - utils - INFO - 1, epoch: 1118, all client loss: [0.5296175479888916, 0.46962693333625793], all pred client disparities: [0.010209918022155762, 0.0004074275493621826], all client disparities: [0.005072444677352905, 0.007061600685119629], all client accs: [0.757869303226471, 0.7779021859169006],  alphas:tensor([0.7642, 0.0000, 0.0000, 0.2358], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:29,646 - utils - INFO - stage2_gradient_single_runtime: 0.00627589225769043
2023-09-28 23:29:29,651 - utils - INFO - 1, epoch: 1119, all client loss: [0.5296381115913391, 0.4693703353404999], all pred client disparities: [0.010266780853271484, 0.00025275349617004395], all client disparities: [0.005072444677352905, 0.007280871272087097], all client accs: [0.757869303226471, 0.7779021859169006],  alphas:tensor([0.4666, 0.0000, 0.1048, 0.4286], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:29,742 - utils - INFO - valid: True, epoch: 1119, loss: [0.5806740522384644, 0.46908965706825256], accuracy: [0.7237569093704224, 0.7791304588317871], mean_accuracy:0.7514436841011047,variance_accuracy:0.027686774730682373, disparity: [0.024242430925369263, 0.01640276610851288], mean_disparity:0.02032259851694107,variance_disparity:0.003919832408428192, pred_disparity: [0.007399559020996094, 0.0028720200061798096]
2023-09-28 23:29:29,868 - utils - INFO - global_valid: True, epoch: 1119,  global_loss: 0.4703301787376404, global_accuracy: 0.810677938175026,  global_disparity:0.01828579604625702, global_pred_disparity: 0.0057954341173172,
2023-09-28 23:29:30,100 - utils - INFO - stage2_gradient_single_runtime: 0.006269216537475586
2023-09-28 23:29:30,105 - utils - INFO - 1, epoch: 1120, all client loss: [0.5296393036842346, 0.46947917342185974], all pred client disparities: [0.010214239358901978, 0.00021579861640930176], all client disparities: [0.005072444677352905, 0.007280871272087097], all client accs: [0.757869303226471, 0.7779021859169006],  alphas:tensor([0.4665, 0.0000, 0.1049, 0.4285], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:30,338 - utils - INFO - stage2_gradient_single_runtime: 0.006211757659912109
2023-09-28 23:29:30,343 - utils - INFO - 1, epoch: 1121, all client loss: [0.529640793800354, 0.4695878028869629], all pred client disparities: [0.010161817073822021, 0.00017862021923065186], all client disparities: [0.005072444677352905, 0.007207781076431274], all client accs: [0.757869303226471, 0.7779332995414734],  alphas:tensor([0.4665, 0.0000, 0.1051, 0.4284], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:30,575 - utils - INFO - stage2_gradient_single_runtime: 0.006237983703613281
2023-09-28 23:29:30,580 - utils - INFO - 1, epoch: 1122, all client loss: [0.5296425819396973, 0.4696963131427765], all pred client disparities: [0.010109364986419678, 0.00014121830463409424], all client disparities: [0.005072444677352905, 0.007134690880775452], all client accs: [0.757869303226471, 0.7779644131660461],  alphas:tensor([0.7636, 0.0000, 0.0000, 0.2364], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:30,812 - utils - INFO - stage2_gradient_single_runtime: 0.00626826286315918
2023-09-28 23:29:30,817 - utils - INFO - 1, epoch: 1123, all client loss: [0.5296626687049866, 0.46943992376327515], all pred client disparities: [0.010166406631469727, 1.2308359146118164e-05], all client disparities: [0.005072444677352905, 0.007427036762237549], all client accs: [0.757869303226471, 0.7779332995414734],  alphas:tensor([-1.5434e-16,  2.3119e-01,  1.9477e-01,  5.7404e-01], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:29:31,050 - utils - INFO - stage2_gradient_single_runtime: 0.00629425048828125
2023-09-28 23:29:31,055 - utils - INFO - 1, epoch: 1124, all client loss: [0.5295947790145874, 0.46953094005584717], all pred client disparities: [0.010072559118270874, 0.00022943317890167236], all client disparities: [0.005072444677352905, 0.007280871272087097], all client accs: [0.757869303226471, 0.7779021859169006],  alphas:tensor([0.4663, 0.0000, 0.1055, 0.4282], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:31,285 - utils - INFO - stage2_gradient_single_runtime: 0.0061969757080078125
2023-09-28 23:29:31,290 - utils - INFO - 1, epoch: 1125, all client loss: [0.5295968651771545, 0.4696390926837921], all pred client disparities: [0.010020166635513306, 0.00019171833992004395], all client disparities: [0.005072444677352905, 0.007207781076431274], all client accs: [0.757869303226471, 0.7779332995414734],  alphas:tensor([0.7637, 0.0000, 0.0000, 0.2363], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:31,525 - utils - INFO - stage2_gradient_single_runtime: 0.006285905838012695
2023-09-28 23:29:31,530 - utils - INFO - 1, epoch: 1126, all client loss: [0.5296170115470886, 0.4693828225135803], all pred client disparities: [0.01007753610610962, 3.802776336669922e-05], all client disparities: [0.005072444677352905, 0.007573217153549194], all client accs: [0.757869303226471, 0.7779021859169006],  alphas:tensor([0.4662, 0.0000, 0.1055, 0.4283], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:31,763 - utils - INFO - stage2_gradient_single_runtime: 0.006234169006347656
2023-09-28 23:29:31,768 - utils - INFO - 1, epoch: 1127, all client loss: [0.5296180248260498, 0.4694916009902954], all pred client disparities: [0.010025262832641602, 1.8627047211339232e-06], all client disparities: [0.005072444677352905, 0.007427036762237549], all client accs: [0.757869303226471, 0.7779332995414734],  alphas:tensor([0.4662, 0.0000, 0.1057, 0.4282], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:32,003 - utils - INFO - stage2_gradient_single_runtime: 0.006257772445678711
2023-09-28 23:29:32,009 - utils - INFO - 1, epoch: 1128, all client loss: [0.5296193361282349, 0.46960020065307617], all pred client disparities: [0.009972959756851196, 3.458559876889922e-05], all client disparities: [0.005072444677352905, 0.007427036762237549], all client accs: [0.757869303226471, 0.7779021859169006],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:32,247 - utils - INFO - stage2_gradient_single_runtime: 0.006228923797607422
2023-09-28 23:29:32,252 - utils - INFO - 1, epoch: 1129, all client loss: [0.5292438864707947, 0.47012874484062195], all pred client disparities: [0.01001712679862976, 0.0006265342235565186], all client disparities: [0.005072444677352905, 0.006696149706840515], all client accs: [0.7554479837417603, 0.7774667143821716],  alphas:tensor([0.7856, 0.0000, 0.0000, 0.2144], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:32,486 - utils - INFO - stage2_gradient_single_runtime: 0.006190299987792969
2023-09-28 23:29:32,491 - utils - INFO - 1, epoch: 1130, all client loss: [0.5292569994926453, 0.46986111998558044], all pred client disparities: [0.010079145431518555, 0.00047647953033447266], all client disparities: [0.005072444677352905, 0.006842330098152161], all client accs: [0.7554479837417603, 0.7777466773986816],  alphas:tensor([0.7803, 0.0000, 0.0000, 0.2197], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:32,722 - utils - INFO - stage2_gradient_single_runtime: 0.0062103271484375
2023-09-28 23:29:32,727 - utils - INFO - 1, epoch: 1131, all client loss: [0.5292714238166809, 0.4695974588394165], all pred client disparities: [0.01014050841331482, 0.00032623112201690674], all client disparities: [0.005072444677352905, 0.006915420293807983], all client accs: [0.7554479837417603, 0.7777466773986816],  alphas:tensor([0.7751, 0.0000, 0.0000, 0.2249], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:32,960 - utils - INFO - stage2_gradient_single_runtime: 0.006331920623779297
2023-09-28 23:29:32,965 - utils - INFO - 1, epoch: 1132, all client loss: [0.5292869806289673, 0.4693377614021301], all pred client disparities: [0.010201305150985718, 0.00017583370208740234], all client disparities: [0.005072444677352905, 0.007134690880775452], all client accs: [0.757869303226471, 0.7778399586677551],  alphas:tensor([0.5215, 0.0000, 0.2851, 0.1934], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:33,196 - utils - INFO - stage2_gradient_single_runtime: 0.0062334537506103516
2023-09-28 23:29:33,202 - utils - INFO - 1, epoch: 1133, all client loss: [0.5292001962661743, 0.4694010317325592], all pred client disparities: [0.010138332843780518, 0.00042378902435302734], all client disparities: [0.005072444677352905, 0.006988510489463806], all client accs: [0.7554479837417603, 0.7777777910232544],  alphas:tensor([0.7742, 0.0000, 0.0000, 0.2258], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:33,442 - utils - INFO - stage2_gradient_single_runtime: 0.006269216537475586
2023-09-28 23:29:33,448 - utils - INFO - 1, epoch: 1134, all client loss: [0.5292161107063293, 0.46914276480674744], all pred client disparities: [0.01019933819770813, 0.0002730637788772583], all client disparities: [0.005072444677352905, 0.007061600685119629], all client accs: [0.757869303226471, 0.7778710722923279],  alphas:tensor([0.4679, 0.0000, 0.1058, 0.4263], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:33,682 - utils - INFO - stage2_gradient_single_runtime: 0.006266117095947266
2023-09-28 23:29:33,687 - utils - INFO - 1, epoch: 1135, all client loss: [0.5292153358459473, 0.4692518413066864], all pred client disparities: [0.010148733854293823, 0.0002386718988418579], all client disparities: [0.005072444677352905, 0.007061600685119629], all client accs: [0.7554479837417603, 0.7778710722923279],  alphas:tensor([0.7707, 0.0000, 0.0000, 0.2293], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:33,918 - utils - INFO - stage2_gradient_single_runtime: 0.006272315979003906
2023-09-28 23:29:33,923 - utils - INFO - 1, epoch: 1136, all client loss: [0.5292317867279053, 0.4689961075782776], all pred client disparities: [0.010209381580352783, 8.83340835571289e-05], all client disparities: [0.005072444677352905, 0.007207781076431274], all client accs: [0.757869303226471, 0.7778399586677551],  alphas:tensor([0.4679, 0.0000, 0.1058, 0.4263], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:34,158 - utils - INFO - stage2_gradient_single_runtime: 0.006278276443481445
2023-09-28 23:29:34,163 - utils - INFO - 1, epoch: 1137, all client loss: [0.529229998588562, 0.4691057801246643], all pred client disparities: [0.010158896446228027, 5.5462121963500977e-05], all client disparities: [0.005072444677352905, 0.007207781076431274], all client accs: [0.7554479837417603, 0.7778399586677551],  alphas:tensor([0.4679, 0.0000, 0.1059, 0.4262], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:34,397 - utils - INFO - stage2_gradient_single_runtime: 0.006266355514526367
2023-09-28 23:29:34,402 - utils - INFO - 1, epoch: 1138, all client loss: [0.5292285680770874, 0.4692152738571167], all pred client disparities: [0.01010856032371521, 2.2307043764158152e-05], all client disparities: [0.005072444677352905, 0.007280871272087097], all client accs: [0.7554479837417603, 0.7778089046478271],  alphas:tensor([0.4679, 0.0000, 0.1060, 0.4261], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:34,637 - utils - INFO - stage2_gradient_single_runtime: 0.006302595138549805
2023-09-28 23:29:34,642 - utils - INFO - 1, epoch: 1139, all client loss: [0.5292274355888367, 0.46932461857795715], all pred client disparities: [0.010058194398880005, 1.1146068572998047e-05], all client disparities: [0.005072444677352905, 0.007280871272087097], all client accs: [0.7554479837417603, 0.7778089046478271],  alphas:tensor([0.5392, 0.0000, 0.2712, 0.1895], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:34,723 - utils - INFO - valid: True, epoch: 1139, loss: [0.5807062983512878, 0.46879568696022034], accuracy: [0.7292817831039429, 0.779254674911499], mean_accuracy:0.754268229007721,variance_accuracy:0.024986445903778076, disparity: [0.009090900421142578, 0.018016979098320007], mean_disparity:0.013553939759731293,variance_disparity:0.004463039338588715, pred_disparity: [0.00652042031288147, 0.003273501992225647]
2023-09-28 23:29:34,851 - utils - INFO - global_valid: True, epoch: 1139,  global_loss: 0.47003987431526184, global_accuracy: 0.8107049533288126,  global_disparity:0.021238043904304504, global_pred_disparity: 0.006230533123016357,
2023-09-28 23:29:35,095 - utils - INFO - stage2_gradient_single_runtime: 0.006250858306884766
2023-09-28 23:29:35,099 - utils - INFO - 1, epoch: 1140, all client loss: [0.5293799638748169, 0.4691874086856842], all pred client disparities: [0.009971201419830322, 0.0002903193235397339], all client disparities: [0.005072444677352905, 0.007427036762237549], all client accs: [0.7554479837417603, 0.7779332995414734],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:35,337 - utils - INFO - stage2_gradient_single_runtime: 0.006217002868652344
2023-09-28 23:29:35,342 - utils - INFO - 1, epoch: 1141, all client loss: [0.5290122628211975, 0.46970832347869873], all pred client disparities: [0.010000854730606079, 0.00036281347274780273], all client disparities: [0.005072444677352905, 0.008356332778930664], all client accs: [0.7554479837417603, 0.7777777910232544],  alphas:tensor([0.7861, 0.0000, 0.0000, 0.2139], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:35,580 - utils - INFO - stage2_gradient_single_runtime: 0.006249904632568359
2023-09-28 23:29:35,585 - utils - INFO - 1, epoch: 1142, all client loss: [0.5290237665176392, 0.46944406628608704], all pred client disparities: [0.010064482688903809, 0.00021548569202423096], all client disparities: [0.005072444677352905, 0.008429422974586487], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.7807, 0.0000, 0.0000, 0.2193], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:35,822 - utils - INFO - stage2_gradient_single_runtime: 0.006238698959350586
2023-09-28 23:29:35,827 - utils - INFO - 1, epoch: 1143, all client loss: [0.5290364623069763, 0.46918371319770813], all pred client disparities: [0.010127604007720947, 6.796419620513916e-05], all client disparities: [0.005072444677352905, 0.008575603365898132], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.5150, 0.0000, 0.2809, 0.2041], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:36,023 - utils - INFO - stage2_gradient_single_runtime: 0.006193399429321289
2023-09-28 23:29:36,029 - utils - INFO - 1, epoch: 1144, all client loss: [0.5289531350135803, 0.46924543380737305], all pred client disparities: [0.010065257549285889, 0.0003070235252380371], all client disparities: [0.005072444677352905, 0.00850251317024231], all client accs: [0.7554479837417603, 0.7778710722923279],  alphas:tensor([0.7798, 0.0000, 0.0000, 0.2202], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:36,264 - utils - INFO - stage2_gradient_single_runtime: 0.006360054016113281
2023-09-28 23:29:36,270 - utils - INFO - 1, epoch: 1145, all client loss: [0.5289661288261414, 0.4689866304397583], all pred client disparities: [0.010128676891326904, 0.0001592189073562622], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.7746, 0.0000, 0.0000, 0.2254], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:36,507 - utils - INFO - stage2_gradient_single_runtime: 0.006254911422729492
2023-09-28 23:29:36,514 - utils - INFO - 1, epoch: 1146, all client loss: [0.5289802551269531, 0.46873167157173157], all pred client disparities: [0.010191380977630615, 1.1369595995347481e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.4688, 0.0000, 0.1063, 0.4249], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:36,747 - utils - INFO - stage2_gradient_single_runtime: 0.006438255310058594
2023-09-28 23:29:36,754 - utils - INFO - 1, epoch: 1147, all client loss: [0.5289767384529114, 0.4688418209552765], all pred client disparities: [0.010141849517822266, 1.9073486328125e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.0000, 0.2470, 0.1948, 0.5582], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:36,992 - utils - INFO - stage2_gradient_single_runtime: 0.006246805191040039
2023-09-28 23:29:36,998 - utils - INFO - 1, epoch: 1148, all client loss: [0.528907060623169, 0.46893227100372314], all pred client disparities: [0.010056018829345703, 0.00021623075008392334], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.7756, 0.0000, 0.0000, 0.2244], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:37,236 - utils - INFO - stage2_gradient_single_runtime: 0.006259441375732422
2023-09-28 23:29:37,241 - utils - INFO - 1, epoch: 1149, all client loss: [0.5289208889007568, 0.4686771631240845], all pred client disparities: [0.010119259357452393, 6.842613220214844e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.4687, 0.0000, 0.1067, 0.4246], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:37,468 - utils - INFO - stage2_gradient_single_runtime: 0.006270170211791992
2023-09-28 23:29:37,472 - utils - INFO - 1, epoch: 1150, all client loss: [0.5289175510406494, 0.4687870442867279], all pred client disparities: [0.010069817304611206, 3.7834051909158006e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.4687, 0.0000, 0.1068, 0.4245], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:37,707 - utils - INFO - stage2_gradient_single_runtime: 0.006250619888305664
2023-09-28 23:29:37,712 - utils - INFO - 1, epoch: 1151, all client loss: [0.5289145708084106, 0.4688967168331146], all pred client disparities: [0.010020464658737183, 6.899253548908746e-06], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.4687, 0.0000, 0.1069, 0.4244], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:37,949 - utils - INFO - stage2_gradient_single_runtime: 0.006224393844604492
2023-09-28 23:29:37,954 - utils - INFO - 1, epoch: 1152, all client loss: [0.5289118885993958, 0.46900615096092224], all pred client disparities: [0.00997135043144226, 2.4378299713134766e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:38,197 - utils - INFO - stage2_gradient_single_runtime: 0.006304740905761719
2023-09-28 23:29:38,202 - utils - INFO - 1, epoch: 1153, all client loss: [0.5288082361221313, 0.46874770522117615], all pred client disparities: [0.010099858045578003, 1.761317253112793e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.4693, 0.0000, 0.1069, 0.4239], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:38,440 - utils - INFO - stage2_gradient_single_runtime: 0.00622105598449707
2023-09-28 23:29:38,445 - utils - INFO - 1, epoch: 1154, all client loss: [0.5288046002388, 0.4688575267791748], all pred client disparities: [0.010050952434539795, 1.2516975402832031e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.5344, 0.0000, 0.2669, 0.1988], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:38,683 - utils - INFO - stage2_gradient_single_runtime: 0.006215095520019531
2023-09-28 23:29:38,688 - utils - INFO - 1, epoch: 1155, all client loss: [0.5289437174797058, 0.4687305688858032], all pred client disparities: [0.009975075721740723, 0.0002752840518951416], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779332995414734],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:38,924 - utils - INFO - stage2_gradient_single_runtime: 0.0062944889068603516
2023-09-28 23:29:38,929 - utils - INFO - 1, epoch: 1156, all client loss: [0.528588593006134, 0.4692447781562805], all pred client disparities: [0.009985417127609253, 0.0003740936517715454], all client disparities: [0.005072444677352905, 0.008429422974586487], all client accs: [0.7530266642570496, 0.7778399586677551],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:39,166 - utils - INFO - stage2_gradient_single_runtime: 0.006356239318847656
2023-09-28 23:29:39,171 - utils - INFO - 1, epoch: 1157, all client loss: [0.5284920334815979, 0.4689786434173584], all pred client disparities: [0.010108321905136108, 0.0004054456949234009], all client disparities: [0.005072444677352905, 0.00850251317024231], all client accs: [0.7554479837417603, 0.7778399586677551],  alphas:tensor([0.7928, 0.0000, 0.0000, 0.2072], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:39,401 - utils - INFO - stage2_gradient_single_runtime: 0.007314205169677734
2023-09-28 23:29:39,406 - utils - INFO - 1, epoch: 1158, all client loss: [0.5285000801086426, 0.4687172472476959], all pred client disparities: [0.010175228118896484, 0.00026206672191619873], all client disparities: [0.005072444677352905, 0.008575603365898132], all client accs: [0.7554479837417603, 0.7778710722923279],  alphas:tensor([0.7875, 0.0000, 0.0000, 0.2125], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:39,648 - utils - INFO - stage2_gradient_single_runtime: 0.007389545440673828
2023-09-28 23:29:39,653 - utils - INFO - 1, epoch: 1159, all client loss: [0.528509259223938, 0.4684597849845886], all pred client disparities: [0.010241568088531494, 0.00011846423149108887], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7778710722923279],  alphas:tensor([0.4707, 0.0000, 0.1070, 0.4223], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:39,795 - utils - INFO - valid: True, epoch: 1159, loss: [0.5804144144058228, 0.46817153692245483], accuracy: [0.7292817831039429, 0.7790683507919312], mean_accuracy:0.754175066947937,variance_accuracy:0.02489328384399414, disparity: [0.009090900421142578, 0.019335314631462097], mean_disparity:0.014213107526302338,variance_disparity:0.0051222071051597595, pred_disparity: [0.005345135927200317, 0.002739131450653076]
2023-09-28 23:29:39,878 - utils - INFO - global_valid: True, epoch: 1159,  global_loss: 0.46941936016082764, global_accuracy: 0.8114762652428091,  global_disparity:0.02249927632510662, global_pred_disparity: 0.0057525187730789185,
2023-09-28 23:29:40,114 - utils - INFO - stage2_gradient_single_runtime: 0.006253242492675781
2023-09-28 23:29:40,121 - utils - INFO - 1, epoch: 1160, all client loss: [0.5285040140151978, 0.4685699939727783], all pred client disparities: [0.010193407535552979, 9.039044380187988e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7778710722923279],  alphas:tensor([0.5023, 0.0000, 0.2717, 0.2261], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:40,357 - utils - INFO - stage2_gradient_single_runtime: 0.006312847137451172
2023-09-28 23:29:40,363 - utils - INFO - 1, epoch: 1161, all client loss: [0.5284262299537659, 0.4686303436756134], all pred client disparities: [0.010131627321243286, 0.00031410157680511475], all client disparities: [0.005072444677352905, 0.00850251317024231], all client accs: [0.7554479837417603, 0.7778710722923279],  alphas:tensor([0.7886, 0.0000, 0.0000, 0.2114], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:40,604 - utils - INFO - stage2_gradient_single_runtime: 0.006365537643432617
2023-09-28 23:29:40,611 - utils - INFO - 1, epoch: 1162, all client loss: [0.5284349918365479, 0.4683729112148285], all pred client disparities: [0.010198503732681274, 0.00017081201076507568], all client disparities: [0.005072444677352905, 0.008721783757209778], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.4707, 0.0000, 0.1073, 0.4219], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:40,850 - utils - INFO - stage2_gradient_single_runtime: 0.0062677860260009766
2023-09-28 23:29:40,857 - utils - INFO - 1, epoch: 1163, all client loss: [0.5284297466278076, 0.4684829115867615], all pred client disparities: [0.010150492191314697, 0.00014288723468780518], all client disparities: [0.005072444677352905, 0.008721783757209778], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.5008, 0.0000, 0.2711, 0.2282], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:41,094 - utils - INFO - stage2_gradient_single_runtime: 0.006232261657714844
2023-09-28 23:29:41,101 - utils - INFO - 1, epoch: 1164, all client loss: [0.528352677822113, 0.46854308247566223], all pred client disparities: [0.010088980197906494, 0.0003648698329925537], all client disparities: [0.005072444677352905, 0.0069398432970047], all client accs: [0.7530266642570496, 0.7872962355613708],  alphas:tensor([0.7898, 0.0000, 0.0000, 0.2102], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:41,339 - utils - INFO - stage2_gradient_single_runtime: 0.006235837936401367
2023-09-28 23:29:41,346 - utils - INFO - 1, epoch: 1165, all client loss: [0.5283610224723816, 0.4682857096195221], all pred client disparities: [0.010156363248825073, 0.00022189319133758545], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779021859169006],  alphas:tensor([0.4708, 0.0000, 0.1076, 0.4216], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:41,582 - utils - INFO - stage2_gradient_single_runtime: 0.006279468536376953
2023-09-28 23:29:41,588 - utils - INFO - 1, epoch: 1166, all client loss: [0.5283557176589966, 0.4683956205844879], all pred client disparities: [0.010108500719070435, 0.00019410252571105957], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7530266642570496, 0.7779021859169006],  alphas:tensor([0.7867, 0.0000, 0.0000, 0.2133], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:41,825 - utils - INFO - stage2_gradient_single_runtime: 0.006289482116699219
2023-09-28 23:29:41,831 - utils - INFO - 1, epoch: 1167, all client loss: [0.5283644199371338, 0.4681406021118164], all pred client disparities: [0.010175585746765137, 5.142391091794707e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7554479837417603, 0.7779332995414734],  alphas:tensor([0.4709, 0.0000, 0.1076, 0.4215], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:42,067 - utils - INFO - stage2_gradient_single_runtime: 0.006639003753662109
2023-09-28 23:29:42,074 - utils - INFO - 1, epoch: 1168, all client loss: [0.5283580422401428, 0.4682510793209076], all pred client disparities: [0.010127633810043335, 2.5138262571999803e-05], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7530266642570496, 0.7779021859169006],  alphas:tensor([0.4709, 0.0000, 0.1077, 0.4213], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:42,314 - utils - INFO - stage2_gradient_single_runtime: 0.006292819976806641
2023-09-28 23:29:42,322 - utils - INFO - 1, epoch: 1169, all client loss: [0.5283520817756653, 0.46836134791374207], all pred client disparities: [0.010079950094223022, 1.5646928659407422e-06], all client disparities: [0.005072444677352905, 0.008648693561553955], all client accs: [0.7530266642570496, 0.7779021859169006],  alphas:tensor([0.5302, 0.0000, 0.2619, 0.2079], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:42,567 - utils - INFO - stage2_gradient_single_runtime: 0.0064258575439453125
2023-09-28 23:29:42,571 - utils - INFO - 1, epoch: 1170, all client loss: [0.5284778475761414, 0.4682444632053375], all pred client disparities: [0.010014325380325317, 0.00024828314781188965], all client disparities: [0.005072444677352905, 0.008721783757209778], all client accs: [0.7530266642570496, 0.7779332995414734],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:42,808 - utils - INFO - stage2_gradient_single_runtime: 0.006193876266479492
2023-09-28 23:29:42,815 - utils - INFO - 1, epoch: 1171, all client loss: [0.5281358361244202, 0.4687519073486328], all pred client disparities: [0.010005027055740356, 0.00039768218994140625], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7873273491859436],  alphas:tensor([0.8026, 0.0000, 0.0000, 0.1974], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:43,062 - utils - INFO - stage2_gradient_single_runtime: 0.006247282028198242
2023-09-28 23:29:43,069 - utils - INFO - 1, epoch: 1172, all client loss: [0.5281402468681335, 0.4684886932373047], all pred client disparities: [0.010074526071548462, 0.00025793910026550293], all client disparities: [0.005072444677352905, 0.006438657641410828], all client accs: [0.7530266642570496, 0.7873273491859436],  alphas:tensor([0.7971, 0.0000, 0.0000, 0.2029], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:43,307 - utils - INFO - stage2_gradient_single_runtime: 0.0063092708587646484
2023-09-28 23:29:43,315 - utils - INFO - 1, epoch: 1173, all client loss: [0.5281457901000977, 0.4682294428348541], all pred client disparities: [0.010143458843231201, 0.00011783838272094727], all client disparities: [0.005072444677352905, 0.006438657641410828], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4958, 0.0000, 0.2660, 0.2382], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:43,560 - utils - INFO - stage2_gradient_single_runtime: 0.006231546401977539
2023-09-28 23:29:43,564 - utils - INFO - 1, epoch: 1174, all client loss: [0.528070867061615, 0.46828943490982056], all pred client disparities: [0.010082274675369263, 0.00033374130725860596], all client disparities: [0.005072444677352905, 0.006438657641410828], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.7962, 0.0000, 0.0000, 0.2038], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:43,800 - utils - INFO - stage2_gradient_single_runtime: 0.006253719329833984
2023-09-28 23:29:43,805 - utils - INFO - 1, epoch: 1175, all client loss: [0.5280766487121582, 0.46803170442581177], all pred client disparities: [0.010151475667953491, 0.00019347667694091797], all client disparities: [0.005072444677352905, 0.006438657641410828], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4719, 0.0000, 0.1083, 0.4198], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:44,042 - utils - INFO - stage2_gradient_single_runtime: 0.006193876266479492
2023-09-28 23:29:44,048 - utils - INFO - 1, epoch: 1176, all client loss: [0.5280700325965881, 0.4681416451931], all pred client disparities: [0.010104238986968994, 0.00016766786575317383], all client disparities: [0.005072444677352905, 0.006438657641410828], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.7931, 0.0000, 0.0000, 0.2069], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:44,284 - utils - INFO - stage2_gradient_single_runtime: 0.006283283233642578
2023-09-28 23:29:44,289 - utils - INFO - 1, epoch: 1177, all client loss: [0.5280762910842896, 0.4678862690925598], all pred client disparities: [0.010173112154006958, 2.765655517578125e-05], all client disparities: [0.005072444677352905, 0.009003713726997375], all client accs: [0.7530266642570496, 0.7780266404151917],  alphas:tensor([0.4720, 0.0000, 0.1082, 0.4197], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:44,527 - utils - INFO - stage2_gradient_single_runtime: 0.006218910217285156
2023-09-28 23:29:44,532 - utils - INFO - 1, epoch: 1178, all client loss: [0.5280686616897583, 0.46799683570861816], all pred client disparities: [0.010125875473022461, 3.3527944651723374e-06], all client disparities: [0.005072444677352905, 0.009003713726997375], all client accs: [0.7530266642570496, 0.7779644131660461],  alphas:tensor([0.4720, 0.0000, 0.1084, 0.4196], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:44,767 - utils - INFO - stage2_gradient_single_runtime: 0.006222963333129883
2023-09-28 23:29:44,772 - utils - INFO - 1, epoch: 1179, all client loss: [0.5280615091323853, 0.46810710430145264], all pred client disparities: [0.01007869839668274, 2.1442776414914988e-05], all client disparities: [0.005072444677352905, 0.009003713726997375], all client accs: [0.7530266642570496, 0.7779644131660461],  alphas:tensor([0.5282, 0.0000, 0.2580, 0.2139], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:44,856 - utils - INFO - valid: True, epoch: 1179, loss: [0.5803364515304565, 0.4675889313220978], accuracy: [0.7292817831039429, 0.7790683507919312], mean_accuracy:0.754175066947937,variance_accuracy:0.02489328384399414, disparity: [0.009090900421142578, 0.019335314631462097], mean_disparity:0.014213107526302338,variance_disparity:0.0051222071051597595, pred_disparity: [0.004468411207199097, 0.002975255250930786]
2023-09-28 23:29:44,982 - utils - INFO - global_valid: True, epoch: 1179,  global_loss: 0.46884238719940186, global_accuracy: 0.8116695427768861,  global_disparity:0.02249927632510662, global_pred_disparity: 0.006025418639183044,
2023-09-28 23:29:45,225 - utils - INFO - stage2_gradient_single_runtime: 0.006235837936401367
2023-09-28 23:29:45,231 - utils - INFO - 1, epoch: 1180, all client loss: [0.5281792879104614, 0.46799609065055847], all pred client disparities: [0.010018676519393921, 0.0002587735652923584], all client disparities: [0.005072444677352905, 0.009003713726997375], all client accs: [0.7530266642570496, 0.7779955267906189],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:45,469 - utils - INFO - stage2_gradient_single_runtime: 0.006412029266357422
2023-09-28 23:29:45,474 - utils - INFO - 1, epoch: 1181, all client loss: [0.5278463363647461, 0.46849972009658813], all pred client disparities: [0.00999787449836731, 0.00038507580757141113], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7872962355613708],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:45,711 - utils - INFO - stage2_gradient_single_runtime: 0.006253719329833984
2023-09-28 23:29:45,717 - utils - INFO - 1, epoch: 1182, all client loss: [0.5277586579322815, 0.46823152899742126], all pred client disparities: [0.01011192798614502, 0.0004058331251144409], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7872962355613708],  alphas:tensor([0.8084, 0.0000, 0.0000, 0.1916], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:45,956 - utils - INFO - stage2_gradient_single_runtime: 0.00628972053527832
2023-09-28 23:29:45,963 - utils - INFO - 1, epoch: 1183, all client loss: [0.5277607440948486, 0.4679700434207916], all pred client disparities: [0.010183066129684448, 0.00026904046535491943], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7872962355613708],  alphas:tensor([0.8030, 0.0000, 0.0000, 0.1970], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:46,196 - utils - INFO - stage2_gradient_single_runtime: 0.006354331970214844
2023-09-28 23:29:46,203 - utils - INFO - 1, epoch: 1184, all client loss: [0.5277639031410217, 0.4677123725414276], all pred client disparities: [0.010253697633743286, 0.0001318603754043579], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4733, 0.0000, 0.1088, 0.4179], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:46,444 - utils - INFO - stage2_gradient_single_runtime: 0.0064220428466796875
2023-09-28 23:29:46,449 - utils - INFO - 1, epoch: 1185, all client loss: [0.5277556777000427, 0.4678226709365845], all pred client disparities: [0.010206848382949829, 0.00010867416858673096], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4901, 0.0000, 0.2589, 0.2510], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:46,683 - utils - INFO - stage2_gradient_single_runtime: 0.006412982940673828
2023-09-28 23:29:46,688 - utils - INFO - 1, epoch: 1186, all client loss: [0.5276827812194824, 0.4678833782672882], all pred client disparities: [0.010145694017410278, 0.00031848251819610596], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7872962355613708],  alphas:tensor([0.8045, 0.0000, 0.0000, 0.1955], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:46,917 - utils - INFO - stage2_gradient_single_runtime: 0.006251335144042969
2023-09-28 23:29:46,922 - utils - INFO - 1, epoch: 1187, all client loss: [0.5276854038238525, 0.46762561798095703], all pred client disparities: [0.01021680235862732, 0.00018174946308135986], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4734, 0.0000, 0.1091, 0.4175], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:47,158 - utils - INFO - stage2_gradient_single_runtime: 0.006272077560424805
2023-09-28 23:29:47,164 - utils - INFO - 1, epoch: 1188, all client loss: [0.5276771187782288, 0.4677357077598572], all pred client disparities: [0.01016998291015625, 0.0001586824655532837], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4889, 0.0000, 0.2581, 0.2530], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:47,395 - utils - INFO - stage2_gradient_single_runtime: 0.006456851959228516
2023-09-28 23:29:47,398 - utils - INFO - 1, epoch: 1189, all client loss: [0.5276046991348267, 0.4677964150905609], all pred client disparities: [0.010109156370162964, 0.0003672391176223755], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7872962355613708],  alphas:tensor([0.8061, 0.0000, 0.0000, 0.1939], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:47,653 - utils - INFO - stage2_gradient_single_runtime: 0.009657144546508789
2023-09-28 23:29:47,659 - utils - INFO - 1, epoch: 1190, all client loss: [0.5276069045066833, 0.4675385653972626], all pred client disparities: [0.010180741548538208, 0.00023096799850463867], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4736, 0.0000, 0.1094, 0.4171], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:47,925 - utils - INFO - stage2_gradient_single_runtime: 0.0062944889068603516
2023-09-28 23:29:47,929 - utils - INFO - 1, epoch: 1191, all client loss: [0.5275986194610596, 0.4676485061645508], all pred client disparities: [0.010134011507034302, 0.00020806491374969482], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.8032, 0.0000, 0.0000, 0.1968], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:48,175 - utils - INFO - stage2_gradient_single_runtime: 0.00628209114074707
2023-09-28 23:29:48,180 - utils - INFO - 1, epoch: 1192, all client loss: [0.5276011824607849, 0.4673928916454315], all pred client disparities: [0.010205447673797607, 7.195770740509033e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4737, 0.0000, 0.1093, 0.4169], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:48,413 - utils - INFO - stage2_gradient_single_runtime: 0.0062487125396728516
2023-09-28 23:29:48,417 - utils - INFO - 1, epoch: 1193, all client loss: [0.5275918841362, 0.4675033688545227], all pred client disparities: [0.010158568620681763, 5.0500038923928514e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4737, 0.0000, 0.1095, 0.4168], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:48,648 - utils - INFO - stage2_gradient_single_runtime: 0.0062329769134521484
2023-09-28 23:29:48,652 - utils - INFO - 1, epoch: 1194, all client loss: [0.5275830626487732, 0.46761348843574524], all pred client disparities: [0.010112017393112183, 2.853572732419707e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4872, 0.0000, 0.2567, 0.2561], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:48,885 - utils - INFO - stage2_gradient_single_runtime: 0.00669550895690918
2023-09-28 23:29:48,891 - utils - INFO - 1, epoch: 1195, all client loss: [0.5275112390518188, 0.46767404675483704], all pred client disparities: [0.010051369667053223, 0.00023482739925384521], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7872651219367981],  alphas:tensor([0.8074, 0.0000, 0.0000, 0.1926], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:49,127 - utils - INFO - stage2_gradient_single_runtime: 0.007349967956542969
2023-09-28 23:29:49,133 - utils - INFO - 1, epoch: 1196, all client loss: [0.5275126695632935, 0.4674166738986969], all pred client disparities: [0.010123461484909058, 9.968876838684082e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4739, 0.0000, 0.1098, 0.4164], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:49,371 - utils - INFO - stage2_gradient_single_runtime: 0.0070531368255615234
2023-09-28 23:29:49,377 - utils - INFO - 1, epoch: 1197, all client loss: [0.5275037884712219, 0.4675266146659851], all pred client disparities: [0.010076969861984253, 7.787346839904785e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.8046, 0.0000, 0.0000, 0.1954], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:49,614 - utils - INFO - stage2_gradient_single_runtime: 0.0076351165771484375
2023-09-28 23:29:49,621 - utils - INFO - 1, epoch: 1198, all client loss: [0.5275055766105652, 0.4672715365886688], all pred client disparities: [0.010148763656616211, 5.704164505004883e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.0000, 0.2684, 0.1932, 0.5383], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:49,858 - utils - INFO - stage2_gradient_single_runtime: 0.007378101348876953
2023-09-28 23:29:49,864 - utils - INFO - 1, epoch: 1199, all client loss: [0.5274267792701721, 0.4673672914505005], all pred client disparities: [0.010072827339172363, 0.00018303096294403076], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4740, 0.0000, 0.1101, 0.4159], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:49,944 - utils - INFO - valid: True, epoch: 1199, loss: [0.5801184177398682, 0.4670681655406952], accuracy: [0.7292817831039429, 0.7873291969299316], mean_accuracy:0.7583054900169373,variance_accuracy:0.029023706912994385, disparity: [0.009090900421142578, 0.0035767704248428345], mean_disparity:0.006333835422992706,variance_disparity:0.002757064998149872, pred_disparity: [0.0034391283988952637, 0.002431809902191162]
2023-09-28 23:29:50,081 - utils - INFO - global_valid: True, epoch: 1199,  global_loss: 0.4683249890804291, global_accuracy: 0.8122078698990339,  global_disparity:0.0071440041065216064, global_pred_disparity: 0.0055342912673950195,
2023-09-28 23:29:50,316 - utils - INFO - stage2_gradient_single_runtime: 0.0062100887298583984
2023-09-28 23:29:50,322 - utils - INFO - 1, epoch: 1200, all client loss: [0.5274181365966797, 0.4674767851829529], all pred client disparities: [0.01002657413482666, 0.0001609325408935547], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.8073, 0.0000, 0.0000, 0.1927], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:50,563 - utils - INFO - stage2_gradient_single_runtime: 0.006337642669677734
2023-09-28 23:29:50,569 - utils - INFO - 1, epoch: 1201, all client loss: [0.5274192690849304, 0.46722084283828735], all pred client disparities: [0.01009899377822876, 2.6509169401833788e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4742, 0.0000, 0.1101, 0.4157], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:50,803 - utils - INFO - stage2_gradient_single_runtime: 0.00629878044128418
2023-09-28 23:29:50,809 - utils - INFO - 1, epoch: 1202, all client loss: [0.5274096727371216, 0.4673309028148651], all pred client disparities: [0.010052472352981567, 5.8710575103759766e-06], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4742, 0.0000, 0.1102, 0.4156], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:51,051 - utils - INFO - stage2_gradient_single_runtime: 0.0063936710357666016
2023-09-28 23:29:51,058 - utils - INFO - 1, epoch: 1203, all client loss: [0.5274004936218262, 0.4674406349658966], all pred client disparities: [0.010006219148635864, 1.5333302144426852e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.8669, 0.0000, 0.0000, 0.1331], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:51,326 - utils - INFO - stage2_gradient_single_runtime: 0.006346702575683594
2023-09-28 23:29:51,330 - utils - INFO - 1, epoch: 1204, all client loss: [0.5272871255874634, 0.46726474165916443], all pred client disparities: [0.010083585977554321, 0.00011375546455383301], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4746, 0.0000, 0.1105, 0.4150], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:51,569 - utils - INFO - stage2_gradient_single_runtime: 0.006257534027099609
2023-09-28 23:29:51,572 - utils - INFO - 1, epoch: 1205, all client loss: [0.5272780656814575, 0.46737414598464966], all pred client disparities: [0.010037481784820557, 9.250640869140625e-05], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.8114, 0.0000, 0.0000, 0.1886], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:51,827 - utils - INFO - stage2_gradient_single_runtime: 0.006282806396484375
2023-09-28 23:29:51,832 - utils - INFO - 1, epoch: 1206, all client loss: [0.5272778272628784, 0.46711766719818115], all pred client disparities: [0.010110527276992798, 4.027784234494902e-05], all client disparities: [0.005072444677352905, 0.0065117329359054565], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.0000, 0.2708, 0.1931, 0.5362], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:52,110 - utils - INFO - stage2_gradient_single_runtime: 0.010443449020385742
2023-09-28 23:29:52,116 - utils - INFO - 1, epoch: 1207, all client loss: [0.5271981954574585, 0.4672139286994934], all pred client disparities: [0.010035574436187744, 0.0002009570598602295], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7874206900596619],  alphas:tensor([0.8117, 0.0000, 0.0000, 0.1883], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:52,393 - utils - INFO - stage2_gradient_single_runtime: 0.010466575622558594
2023-09-28 23:29:52,399 - utils - INFO - 1, epoch: 1208, all client loss: [0.5271979570388794, 0.46695810556411743], all pred client disparities: [0.010108768939971924, 6.817281246185303e-05], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7874206900596619],  alphas:tensor([0.4749, 0.0000, 0.1107, 0.4144], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:52,670 - utils - INFO - stage2_gradient_single_runtime: 0.006356954574584961
2023-09-28 23:29:52,675 - utils - INFO - 1, epoch: 1209, all client loss: [0.527187705039978, 0.4670679569244385], all pred client disparities: [0.010062426328659058, 4.863739013671875e-05], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7874206900596619],  alphas:tensor([0.4749, 0.0000, 0.1108, 0.4143], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:52,910 - utils - INFO - stage2_gradient_single_runtime: 0.006308555603027344
2023-09-28 23:29:52,915 - utils - INFO - 1, epoch: 1210, all client loss: [0.5271780490875244, 0.46717748045921326], all pred client disparities: [0.010016322135925293, 2.8505925001809373e-05], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7874206900596619],  alphas:tensor([0.4749, 0.0000, 0.1109, 0.4142], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:53,100 - utils - INFO - stage2_gradient_single_runtime: 0.006260871887207031
2023-09-28 23:29:53,105 - utils - INFO - 1, epoch: 1211, all client loss: [0.5271688103675842, 0.467286616563797], all pred client disparities: [0.009970426559448242, 7.808208465576172e-06], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:53,339 - utils - INFO - stage2_gradient_single_runtime: 0.006257534027099609
2023-09-28 23:29:53,344 - utils - INFO - 1, epoch: 1212, all client loss: [0.5270864963531494, 0.46702292561531067], all pred client disparities: [0.0100783109664917, 2.4706125259399414e-05], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7874206900596619],  alphas:tensor([0.4753, 0.0000, 0.1111, 0.4136], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:53,576 - utils - INFO - stage2_gradient_single_runtime: 0.006350278854370117
2023-09-28 23:29:53,581 - utils - INFO - 1, epoch: 1213, all client loss: [0.5270766615867615, 0.46713224053382874], all pred client disparities: [0.010032355785369873, 4.902504770143423e-06], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4822, 0.0000, 0.2488, 0.2690], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:53,816 - utils - INFO - stage2_gradient_single_runtime: 0.006309986114501953
2023-09-28 23:29:53,821 - utils - INFO - 1, epoch: 1214, all client loss: [0.5270066857337952, 0.46719396114349365], all pred client disparities: [0.00997212529182434, 0.00020606815814971924], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:54,053 - utils - INFO - stage2_gradient_single_runtime: 0.0062465667724609375
2023-09-28 23:29:54,058 - utils - INFO - 1, epoch: 1215, all client loss: [0.5269266366958618, 0.4669288992881775], all pred client disparities: [0.010078638792037964, 0.00021946430206298828], all client disparities: [0.005072444677352905, 0.006657913327217102], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.8182, 0.0000, 0.0000, 0.1818], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:54,289 - utils - INFO - stage2_gradient_single_runtime: 0.0062830448150634766
2023-09-28 23:29:54,294 - utils - INFO - 1, epoch: 1216, all client loss: [0.5269245505332947, 0.46667271852493286], all pred client disparities: [0.0101529061794281, 8.884072303771973e-05], all client disparities: [0.005072444677352905, 0.006584823131561279], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4758, 0.0000, 0.1115, 0.4127], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:54,526 - utils - INFO - stage2_gradient_single_runtime: 0.006218433380126953
2023-09-28 23:29:54,531 - utils - INFO - 1, epoch: 1217, all client loss: [0.5269137024879456, 0.4667823016643524], all pred client disparities: [0.010106533765792847, 7.064640522003174e-05], all client disparities: [0.005072444677352905, 0.006584823131561279], all client accs: [0.7530266642570496, 0.7873895764350891],  alphas:tensor([0.4758, 0.0000, 0.1116, 0.4126], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:54,765 - utils - INFO - stage2_gradient_single_runtime: 0.006246805191040039
2023-09-28 23:29:54,770 - utils - INFO - 1, epoch: 1218, all client loss: [0.5269033908843994, 0.46689140796661377], all pred client disparities: [0.010060429573059082, 5.179643630981445e-05], all client disparities: [0.005072444677352905, 0.005937457084655762], all client accs: [0.7530266642570496, 0.7874206900596619],  alphas:tensor([0.4758, 0.0000, 0.1117, 0.4125], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:55,001 - utils - INFO - stage2_gradient_single_runtime: 0.006299734115600586
2023-09-28 23:29:55,006 - utils - INFO - 1, epoch: 1219, all client loss: [0.5268935561180115, 0.46700015664100647], all pred client disparities: [0.01001468300819397, 3.2335519790649414e-05], all client disparities: [0.005072444677352905, 0.0060105472803115845], all client accs: [0.7530266642570496, 0.7874206900596619],  alphas:tensor([0.8210, 0.0000, 0.0000, 0.1790], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:55,149 - utils - INFO - valid: True, epoch: 1219, loss: [0.5799409747123718, 0.4663147032260895], accuracy: [0.7292817831039429, 0.7876397371292114], mean_accuracy:0.7584607601165771,variance_accuracy:0.029178977012634277, disparity: [0.009090900421142578, 0.0035767704248428345], mean_disparity:0.006333835422992706,variance_disparity:0.002757064998149872, pred_disparity: [0.002431035041809082, 0.002522110939025879]
2023-09-28 23:29:55,227 - utils - INFO - global_valid: True, epoch: 1219,  global_loss: 0.46757790446281433, global_accuracy: 0.8126331285659951,  global_disparity:0.0071440041065216064, global_pred_disparity: 0.005664736032485962,
2023-09-28 23:29:55,461 - utils - INFO - stage2_gradient_single_runtime: 0.006270408630371094
2023-09-28 23:29:55,466 - utils - INFO - 1, epoch: 1220, all client loss: [0.5268903970718384, 0.46674275398254395], all pred client disparities: [0.010089129209518433, 9.697675704956055e-05], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7874206900596619],  alphas:tensor([0.9628, 0.0000, 0.0372, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:55,697 - utils - INFO - stage2_gradient_single_runtime: 0.006212711334228516
2023-09-28 23:29:55,702 - utils - INFO - 1, epoch: 1221, all client loss: [0.5266010165214539, 0.4672017991542816], all pred client disparities: [0.010032296180725098, 0.000515669584274292], all client disparities: [0.005072444677352905, 0.0060105472803115845], all client accs: [0.7530266642570496, 0.7872340679168701],  alphas:tensor([0.8403, 0.0000, 0.0000, 0.1597], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:55,934 - utils - INFO - stage2_gradient_single_runtime: 0.0062656402587890625
2023-09-28 23:29:55,939 - utils - INFO - 1, epoch: 1222, all client loss: [0.5265939235687256, 0.466935932636261], all pred client disparities: [0.010108500719070435, 0.0003893822431564331], all client disparities: [0.005072444677352905, 0.0060105472803115845], all client accs: [0.7530266642570496, 0.7872651219367981],  alphas:tensor([0.8345, 0.0000, 0.0000, 0.1655], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:56,175 - utils - INFO - stage2_gradient_single_runtime: 0.006184577941894531
2023-09-28 23:29:56,180 - utils - INFO - 1, epoch: 1223, all client loss: [0.5265880227088928, 0.46667400002479553], all pred client disparities: [0.010184317827224731, 0.00026237964630126953], all client disparities: [0.005072444677352905, 0.005937457084655762], all client accs: [0.7530266642570496, 0.7872651219367981],  alphas:tensor([0.8288, 0.0000, 0.0000, 0.1712], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:56,417 - utils - INFO - stage2_gradient_single_runtime: 0.006272077560424805
2023-09-28 23:29:56,422 - utils - INFO - 1, epoch: 1224, all client loss: [0.5265833735466003, 0.4664159417152405], all pred client disparities: [0.0102597177028656, 0.00013475120067596436], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7873273491859436],  alphas:tensor([0.4770, 0.0000, 0.1124, 0.4105], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:56,660 - utils - INFO - stage2_gradient_single_runtime: 0.0062868595123291016
2023-09-28 23:29:56,665 - utils - INFO - 1, epoch: 1225, all client loss: [0.5265721678733826, 0.46652480959892273], all pred client disparities: [0.010213285684585571, 0.00011761486530303955], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7873273491859436],  alphas:tensor([0.4770, 0.0000, 0.1125, 0.4104], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:56,905 - utils - INFO - stage2_gradient_single_runtime: 0.006304740905761719
2023-09-28 23:29:56,910 - utils - INFO - 1, epoch: 1226, all client loss: [0.5265614986419678, 0.4666332006454468], all pred client disparities: [0.010167181491851807, 9.974837303161621e-05], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7872340679168701],  alphas:tensor([0.4800, 0.0000, 0.2403, 0.2797], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:57,156 - utils - INFO - stage2_gradient_single_runtime: 0.0065288543701171875
2023-09-28 23:29:57,161 - utils - INFO - 1, epoch: 1227, all client loss: [0.5264915227890015, 0.46669819951057434], all pred client disparities: [0.010105520486831665, 0.00030128657817840576], all client disparities: [0.005072444677352905, 0.005937457084655762], all client accs: [0.7530266642570496, 0.7872340679168701],  alphas:tensor([0.8341, 0.0000, 0.0000, 0.1659], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:57,450 - utils - INFO - stage2_gradient_single_runtime: 0.006326436996459961
2023-09-28 23:29:57,455 - utils - INFO - 1, epoch: 1228, all client loss: [0.5264855027198792, 0.46643802523612976], all pred client disparities: [0.010181546211242676, 0.00017483532428741455], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7872340679168701],  alphas:tensor([0.4772, 0.0000, 0.1129, 0.4099], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:57,696 - utils - INFO - stage2_gradient_single_runtime: 0.0095977783203125
2023-09-28 23:29:57,702 - utils - INFO - 1, epoch: 1229, all client loss: [0.5264749526977539, 0.46654611825942993], all pred client disparities: [0.010135531425476074, 0.00015701353549957275], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7872340679168701],  alphas:tensor([0.8313, 0.0000, 0.0000, 0.1687], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:57,962 - utils - INFO - stage2_gradient_single_runtime: 0.006283998489379883
2023-09-28 23:29:57,967 - utils - INFO - 1, epoch: 1230, all client loss: [0.5264694094657898, 0.4662880599498749], all pred client disparities: [0.010211318731307983, 3.0606985092163086e-05], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4774, 0.0000, 0.1129, 0.4097], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:58,196 - utils - INFO - stage2_gradient_single_runtime: 0.006219625473022461
2023-09-28 23:29:58,201 - utils - INFO - 1, epoch: 1231, all client loss: [0.5264580249786377, 0.46639662981033325], all pred client disparities: [0.01016494631767273, 1.4081605513638351e-05], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7873273491859436],  alphas:tensor([0.4774, 0.0000, 0.1130, 0.4096], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:58,432 - utils - INFO - stage2_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:29:58,437 - utils - INFO - 1, epoch: 1232, all client loss: [0.5264471769332886, 0.4665047526359558], all pred client disparities: [0.010119020938873291, 3.129243850708008e-06], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7872340679168701],  alphas:tensor([0.5237, 0.0000, 0.2386, 0.2377], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:58,668 - utils - INFO - stage2_gradient_single_runtime: 0.006258964538574219
2023-09-28 23:29:58,673 - utils - INFO - 1, epoch: 1233, all client loss: [0.5265310406684875, 0.4664197266101837], all pred client disparities: [0.010080099105834961, 0.000201493501663208], all client disparities: [0.005072444677352905, 0.005498930811882019], all client accs: [0.7530266642570496, 0.7878561615943909],  alphas:tensor([0.9481, 0.0000, 0.0519, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:58,902 - utils - INFO - stage2_gradient_single_runtime: 0.006185293197631836
2023-09-28 23:29:58,907 - utils - INFO - 1, epoch: 1234, all client loss: [0.5262552499771118, 0.46686261892318726], all pred client disparities: [0.010015934705734253, 0.0003976672887802124], all client disparities: [0.005072444677352905, 0.003274887800216675], all client accs: [0.7506053447723389, 0.7872651219367981],  alphas:tensor([0.8500, 0.0000, 0.0000, 0.1500], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:59,138 - utils - INFO - stage2_gradient_single_runtime: 0.006168842315673828
2023-09-28 23:29:59,143 - utils - INFO - 1, epoch: 1235, all client loss: [0.526245653629303, 0.46659570932388306], all pred client disparities: [0.010093122720718384, 0.0002748072147369385], all client disparities: [0.005072444677352905, 0.002982527017593384], all client accs: [0.7506053447723389, 0.7872029542922974],  alphas:tensor([0.8440, 0.0000, 0.0000, 0.1560], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:59,368 - utils - INFO - stage2_gradient_single_runtime: 0.006221294403076172
2023-09-28 23:29:59,373 - utils - INFO - 1, epoch: 1236, all client loss: [0.5262373089790344, 0.4663327634334564], all pred client disparities: [0.010170012712478638, 0.00015112757682800293], all client disparities: [0.005072444677352905, 0.005864366888999939], all client accs: [0.7530266642570496, 0.7871718406677246],  alphas:tensor([0.4791, 0.0000, 0.2357, 0.2853], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:59,603 - utils - INFO - stage2_gradient_single_runtime: 0.006210803985595703
2023-09-28 23:29:59,608 - utils - INFO - 1, epoch: 1237, all client loss: [0.5261673927307129, 0.46639958024024963], all pred client disparities: [0.010107725858688354, 0.00035312771797180176], all client disparities: [0.005072444677352905, 0.002408236265182495], all client accs: [0.7506053447723389, 0.7872962355613708],  alphas:tensor([0.8436, 0.0000, 0.0000, 0.1564], device='cuda:0', dtype=torch.float64)
2023-09-28 23:29:59,836 - utils - INFO - stage2_gradient_single_runtime: 0.0062215328216552734
2023-09-28 23:29:59,842 - utils - INFO - 1, epoch: 1238, all client loss: [0.5261592268943787, 0.46613794565200806], all pred client disparities: [0.010184764862060547, 0.00022935867309570312], all client disparities: [0.005072444677352905, 0.002408236265182495], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4780, 0.0000, 0.1142, 0.4078], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:00,071 - utils - INFO - stage2_gradient_single_runtime: 0.0061626434326171875
2023-09-28 23:30:00,076 - utils - INFO - 1, epoch: 1239, all client loss: [0.5261486768722534, 0.46624499559402466], all pred client disparities: [0.010138928890228271, 0.00021207332611083984], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7506053447723389, 0.7873895764350891],  alphas:tensor([0.8408, 0.0000, 0.0000, 0.1592], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:00,156 - utils - INFO - valid: True, epoch: 1239, loss: [0.5796808004379272, 0.4655398726463318], accuracy: [0.7292817831039429, 0.7877018451690674], mean_accuracy:0.7584918141365051,variance_accuracy:0.029210031032562256, disparity: [0.009090900421142578, 0.006361380219459534], mean_disparity:0.007726140320301056,variance_disparity:0.0013647601008415222, pred_disparity: [0.0014345049858093262, 0.002167746424674988]
2023-09-28 23:30:00,297 - utils - INFO - global_valid: True, epoch: 1239,  global_loss: 0.46680882573127747, global_accuracy: 0.8134001853841746,  global_disparity:0.009809732437133789, global_pred_disparity: 0.005355283617973328,
2023-09-28 23:30:00,526 - utils - INFO - stage2_gradient_single_runtime: 0.006260871887207031
2023-09-28 23:30:00,532 - utils - INFO - 1, epoch: 1240, all client loss: [0.5261410474777222, 0.4659854471683502], all pred client disparities: [0.010215729475021362, 8.825957775115967e-05], all client disparities: [0.005072444677352905, 0.0023351460695266724], all client accs: [0.7530266642570496, 0.7873584628105164],  alphas:tensor([0.4782, 0.0000, 0.1142, 0.4076], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:00,764 - utils - INFO - stage2_gradient_single_runtime: 0.006254434585571289
2023-09-28 23:30:00,769 - utils - INFO - 1, epoch: 1241, all client loss: [0.5261296033859253, 0.466092973947525], all pred client disparities: [0.010169416666030884, 7.228553295135498e-05], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7506053447723389, 0.7873895764350891],  alphas:tensor([0.4782, 0.0000, 0.1143, 0.4075], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:00,999 - utils - INFO - stage2_gradient_single_runtime: 0.0062274932861328125
2023-09-28 23:30:01,005 - utils - INFO - 1, epoch: 1242, all client loss: [0.5261187553405762, 0.466200053691864], all pred client disparities: [0.010123461484909058, 5.558133125305176e-05], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7506053447723389, 0.7873895764350891],  alphas:tensor([0.4784, 0.0000, 0.2341, 0.2875], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:01,235 - utils - INFO - stage2_gradient_single_runtime: 0.006325483322143555
2023-09-28 23:30:01,239 - utils - INFO - 1, epoch: 1243, all client loss: [0.526049017906189, 0.46626725792884827], all pred client disparities: [0.010061264038085938, 0.000257149338722229], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7506053447723389, 0.7873584628105164],  alphas:tensor([0.8466, 0.0000, 0.0000, 0.1534], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:01,471 - utils - INFO - stage2_gradient_single_runtime: 0.006292819976806641
2023-09-28 23:30:01,476 - utils - INFO - 1, epoch: 1244, all client loss: [0.5260400772094727, 0.46600541472435], all pred client disparities: [0.010138630867004395, 0.00013457238674163818], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7506053447723389, 0.7874206900596619],  alphas:tensor([0.4783, 0.0000, 0.1148, 0.4070], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:01,709 - utils - INFO - stage2_gradient_single_runtime: 0.006228208541870117
2023-09-28 23:30:01,714 - utils - INFO - 1, epoch: 1245, all client loss: [0.5260294675827026, 0.46611207723617554], all pred client disparities: [0.01009279489517212, 0.00011770427227020264], all client disparities: [0.005072444677352905, 0.0025544166564941406], all client accs: [0.7506053447723389, 0.7874206900596619],  alphas:tensor([0.8437, 0.0000, 0.0000, 0.1563], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:01,943 - utils - INFO - stage2_gradient_single_runtime: 0.00625157356262207
2023-09-28 23:30:01,947 - utils - INFO - 1, epoch: 1246, all client loss: [0.5260209441184998, 0.46585240960121155], all pred client disparities: [0.010169953107833862, 4.902504770143423e-06], all client disparities: [0.005072444677352905, 0.0019697099924087524], all client accs: [0.7506053447723389, 0.7878872752189636],  alphas:tensor([0.4798, 0.3514, 0.1688, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:02,179 - utils - INFO - stage2_gradient_single_runtime: 0.00625300407409668
2023-09-28 23:30:02,185 - utils - INFO - 1, epoch: 1247, all client loss: [0.5259876847267151, 0.46595796942710876], all pred client disparities: [0.010131120681762695, 2.3320322725339793e-05], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7506053447723389, 0.7874518036842346],  alphas:tensor([0.4785, 0.0000, 0.1150, 0.4065], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:02,417 - utils - INFO - stage2_gradient_single_runtime: 0.0062367916107177734
2023-09-28 23:30:02,422 - utils - INFO - 1, epoch: 1248, all client loss: [0.5259768962860107, 0.4660644829273224], all pred client disparities: [0.01008528470993042, 6.839649358880706e-06], all client disparities: [0.005072444677352905, 0.0025544166564941406], all client accs: [0.7506053447723389, 0.7874518036842346],  alphas:tensor([0.4780, 0.0000, 0.2322, 0.2898], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:02,653 - utils - INFO - stage2_gradient_single_runtime: 0.0062296390533447266
2023-09-28 23:30:02,657 - utils - INFO - 1, epoch: 1249, all client loss: [0.5259073972702026, 0.4661322832107544], all pred client disparities: [0.010022938251495361, 0.00020836293697357178], all client disparities: [0.005072444677352905, 0.0025544166564941406], all client accs: [0.7506053447723389, 0.7873584628105164],  alphas:tensor([0.8508, 0.0000, 0.0000, 0.1492], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:02,887 - utils - INFO - stage2_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:30:02,893 - utils - INFO - 1, epoch: 1250, all client loss: [0.5258973836898804, 0.46586984395980835], all pred client disparities: [0.010100722312927246, 8.71419906616211e-05], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7506053447723389, 0.7874518036842346],  alphas:tensor([0.4786, 0.0000, 0.1154, 0.4060], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:03,121 - utils - INFO - stage2_gradient_single_runtime: 0.006222724914550781
2023-09-28 23:30:03,126 - utils - INFO - 1, epoch: 1251, all client loss: [0.5258867740631104, 0.46597591042518616], all pred client disparities: [0.010055065155029297, 7.046759128570557e-05], all client disparities: [0.005072444677352905, 0.0025544166564941406], all client accs: [0.7506053447723389, 0.7874518036842346],  alphas:tensor([0.8479, 0.0000, 0.0000, 0.1521], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:03,355 - utils - INFO - stage2_gradient_single_runtime: 0.006241559982299805
2023-09-28 23:30:03,360 - utils - INFO - 1, epoch: 1252, all client loss: [0.5258773565292358, 0.46571558713912964], all pred client disparities: [0.010132640600204468, 5.088746911496855e-05], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7506053447723389, 0.7879183888435364],  alphas:tensor([0.4810, 0.3493, 0.1697, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:03,591 - utils - INFO - stage2_gradient_single_runtime: 0.006223440170288086
2023-09-28 23:30:03,596 - utils - INFO - 1, epoch: 1253, all client loss: [0.5258439779281616, 0.4658210277557373], all pred client disparities: [0.010093599557876587, 2.1338462829589844e-05], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7506053447723389, 0.7879183888435364],  alphas:tensor([0.4803, 0.3495, 0.1702, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:03,827 - utils - INFO - stage2_gradient_single_runtime: 0.0063381195068359375
2023-09-28 23:30:03,832 - utils - INFO - 1, epoch: 1254, all client loss: [0.5258116722106934, 0.4659254252910614], all pred client disparities: [0.010055065155029297, 6.571428912138799e-06], all client disparities: [0.005072444677352905, 0.002188965678215027], all client accs: [0.7506053447723389, 0.7879183888435364],  alphas:tensor([0.4778, 0.0000, 0.2301, 0.2921], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:04,064 - utils - INFO - stage2_gradient_single_runtime: 0.006247758865356445
2023-09-28 23:30:04,069 - utils - INFO - 1, epoch: 1255, all client loss: [0.5257421731948853, 0.4659939408302307], all pred client disparities: [0.009992420673370361, 0.0002084970474243164], all client disparities: [0.005072444677352905, 0.0025544166564941406], all client accs: [0.7506053447723389, 0.7873584628105164],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:04,301 - utils - INFO - stage2_gradient_single_runtime: 0.006234169006347656
2023-09-28 23:30:04,306 - utils - INFO - 1, epoch: 1256, all client loss: [0.5256765484809875, 0.46572378277778625], all pred client disparities: [0.010089337825775146, 0.00020009279251098633], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7506053447723389, 0.7874206900596619],  alphas:tensor([0.8531, 0.0000, 0.0000, 0.1469], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:04,537 - utils - INFO - stage2_gradient_single_runtime: 0.006251811981201172
2023-09-28 23:30:04,543 - utils - INFO - 1, epoch: 1257, all client loss: [0.5256662368774414, 0.46546313166618347], all pred client disparities: [0.01016739010810852, 7.987022399902344e-05], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7506053447723389, 0.7878872752189636],  alphas:tensor([0.4792, 0.0000, 0.1164, 0.4044], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:04,785 - utils - INFO - stage2_gradient_single_runtime: 0.006262063980102539
2023-09-28 23:30:04,790 - utils - INFO - 1, epoch: 1258, all client loss: [0.5256549715995789, 0.4655689299106598], all pred client disparities: [0.01012122631072998, 6.471574306488037e-05], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7506053447723389, 0.7878872752189636],  alphas:tensor([0.4792, 0.0000, 0.1165, 0.4043], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:05,022 - utils - INFO - stage2_gradient_single_runtime: 0.006281375885009766
2023-09-28 23:30:05,027 - utils - INFO - 1, epoch: 1259, all client loss: [0.5256441831588745, 0.4656742513179779], all pred client disparities: [0.010075360536575317, 4.874170190305449e-05], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7506053447723389, 0.7878872752189636],  alphas:tensor([0.4773, 0.0000, 0.2283, 0.2944], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:05,108 - utils - INFO - valid: True, epoch: 1259, loss: [0.5796409249305725, 0.46530044078826904], accuracy: [0.7292817831039429, 0.7877639532089233], mean_accuracy:0.7585228681564331,variance_accuracy:0.029241085052490234, disparity: [0.009090900421142578, 0.006213456392288208], mean_disparity:0.007652178406715393,variance_disparity:0.001438722014427185, pred_disparity: [0.0005593299865722656, 0.0018778294324874878]
2023-09-28 23:30:05,233 - utils - INFO - global_valid: True, epoch: 1259,  global_loss: 0.466571569442749, global_accuracy: 0.8138438758298221,  global_disparity:0.009666457772254944, global_pred_disparity: 0.005106478929519653,
2023-09-28 23:30:05,469 - utils - INFO - stage2_gradient_single_runtime: 0.0063402652740478516
2023-09-28 23:30:05,474 - utils - INFO - 1, epoch: 1260, all client loss: [0.5255744457244873, 0.4657440185546875], all pred client disparities: [0.01001209020614624, 0.00025169551372528076], all client disparities: [0.005072444677352905, 0.002481326460838318], all client accs: [0.7457627654075623, 0.7873584628105164],  alphas:tensor([0.8593, 0.0000, 0.0000, 0.1407], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:05,715 - utils - INFO - stage2_gradient_single_runtime: 0.006268739700317383
2023-09-28 23:30:05,719 - utils - INFO - 1, epoch: 1261, all client loss: [0.5255628228187561, 0.4654809236526489], all pred client disparities: [0.010090738534927368, 0.0001327991485595703], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7506053447723389, 0.7878872752189636],  alphas:tensor([0.4792, 0.0000, 0.1170, 0.4038], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:05,907 - utils - INFO - stage2_gradient_single_runtime: 0.007563591003417969
2023-09-28 23:30:05,913 - utils - INFO - 1, epoch: 1262, all client loss: [0.5255523324012756, 0.46558573842048645], all pred client disparities: [0.010045051574707031, 0.00011660158634185791], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7457627654075623, 0.7878561615943909],  alphas:tensor([0.8564, 0.0000, 0.0000, 0.1436], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:06,205 - utils - INFO - stage2_gradient_single_runtime: 0.006220340728759766
2023-09-28 23:30:06,209 - utils - INFO - 1, epoch: 1263, all client loss: [0.5255412459373474, 0.4653247892856598], all pred client disparities: [0.010123491287231445, 2.5033950805664062e-06], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7506053447723389, 0.7879183888435364],  alphas:tensor([0.4839, 0.3444, 0.1717, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:06,445 - utils - INFO - stage2_gradient_single_runtime: 0.006295204162597656
2023-09-28 23:30:06,450 - utils - INFO - 1, epoch: 1264, all client loss: [0.5255072116851807, 0.4654303789138794], all pred client disparities: [0.01008373498916626, 3.062188989133574e-05], all client disparities: [0.005072444677352905, 0.002042800188064575], all client accs: [0.7457627654075623, 0.7879183888435364],  alphas:tensor([0.4794, 0.0000, 0.1172, 0.4033], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:06,685 - utils - INFO - stage2_gradient_single_runtime: 0.006317615509033203
2023-09-28 23:30:06,691 - utils - INFO - 1, epoch: 1265, all client loss: [0.5254966616630554, 0.4655349552631378], all pred client disparities: [0.010038107633590698, 1.4618046407122165e-05], all client disparities: [0.005072444677352905, 0.002115890383720398], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.4772, 0.0000, 0.2266, 0.2962], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:06,926 - utils - INFO - stage2_gradient_single_runtime: 0.006246805191040039
2023-09-28 23:30:06,931 - utils - INFO - 1, epoch: 1266, all client loss: [0.5254271030426025, 0.46560537815093994], all pred client disparities: [0.00997459888458252, 0.00021797418594360352], all client disparities: [0.005072444677352905, 0.002700597047805786], all client accs: [0.7457627654075623, 0.7873895764350891],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:07,165 - utils - INFO - stage2_gradient_single_runtime: 0.0062673091888427734
2023-09-28 23:30:07,170 - utils - INFO - 1, epoch: 1267, all client loss: [0.5253645181655884, 0.4653348922729492], all pred client disparities: [0.010069787502288818, 0.0002048015594482422], all client disparities: [0.005072444677352905, 0.0022620558738708496], all client accs: [0.7457627654075623, 0.7879183888435364],  alphas:tensor([0.4795, 0.0000, 0.1179, 0.4025], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:07,404 - utils - INFO - stage2_gradient_single_runtime: 0.006243705749511719
2023-09-28 23:30:07,409 - utils - INFO - 1, epoch: 1268, all client loss: [0.5253545045852661, 0.4654385447502136], all pred client disparities: [0.010024398565292358, 0.00018806755542755127], all client disparities: [0.005072444677352905, 0.0022620558738708496], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.8637, 0.0000, 0.0000, 0.1363], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:07,669 - utils - INFO - stage2_gradient_single_runtime: 0.006337881088256836
2023-09-28 23:30:07,674 - utils - INFO - 1, epoch: 1269, all client loss: [0.5253421068191528, 0.46517589688301086], all pred client disparities: [0.010103464126586914, 7.06017017364502e-05], all client disparities: [0.005072444677352905, 0.002188965678215027], all client accs: [0.7457627654075623, 0.7879805564880371],  alphas:tensor([0.4797, 0.0000, 0.1180, 0.4023], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:07,905 - utils - INFO - stage2_gradient_single_runtime: 0.00632476806640625
2023-09-28 23:30:07,910 - utils - INFO - 1, epoch: 1270, all client loss: [0.525331437587738, 0.4652799963951111], all pred client disparities: [0.010057628154754639, 5.510449409484863e-05], all client disparities: [0.005072444677352905, 0.0022620558738708496], all client accs: [0.7457627654075623, 0.7879183888435364],  alphas:tensor([0.4797, 0.0000, 0.1181, 0.4022], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:08,143 - utils - INFO - stage2_gradient_single_runtime: 0.006243228912353516
2023-09-28 23:30:08,148 - utils - INFO - 1, epoch: 1271, all client loss: [0.5253212451934814, 0.4653835594654083], all pred client disparities: [0.010012120008468628, 3.8787726225564256e-05], all client disparities: [0.005072444677352905, 0.0022620558738708496], all client accs: [0.7457627654075623, 0.7879183888435364],  alphas:tensor([0.8639, 0.0000, 0.0000, 0.1361], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:08,380 - utils - INFO - stage2_gradient_single_runtime: 0.006297111511230469
2023-09-28 23:30:08,386 - utils - INFO - 1, epoch: 1272, all client loss: [0.5253086686134338, 0.465121328830719], all pred client disparities: [0.010091155767440796, 7.821619510650635e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7880427837371826],  alphas:tensor([0.4857, 0.3411, 0.1732, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:08,616 - utils - INFO - stage2_gradient_single_runtime: 0.006281852722167969
2023-09-28 23:30:08,622 - utils - INFO - 1, epoch: 1273, all client loss: [0.5252746939659119, 0.46522650122642517], all pred client disparities: [0.010051131248474121, 4.336237907409668e-05], all client disparities: [0.005072444677352905, 0.002188965678215027], all client accs: [0.7457627654075623, 0.7879805564880371],  alphas:tensor([0.4848, 0.3415, 0.1737, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:08,851 - utils - INFO - stage2_gradient_single_runtime: 0.0062541961669921875
2023-09-28 23:30:08,856 - utils - INFO - 1, epoch: 1274, all client loss: [0.5252418518066406, 0.46533042192459106], all pred client disparities: [0.010011613368988037, 1.0460615158081055e-05], all client disparities: [0.005072444677352905, 0.0022620558738708496], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.5251, 0.0000, 0.2247, 0.2502], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:09,089 - utils - INFO - stage2_gradient_single_runtime: 0.006277561187744141
2023-09-28 23:30:09,094 - utils - INFO - 1, epoch: 1275, all client loss: [0.5253089070320129, 0.46525827050209045], all pred client disparities: [0.009980380535125732, 0.00019086897373199463], all client disparities: [0.005072444677352905, 0.002836361527442932], all client accs: [0.7457627654075623, 0.7880116701126099],  alphas:tensor([0.8996, 0.0000, 0.1004, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:09,323 - utils - INFO - stage2_gradient_single_runtime: 0.006257534027099609
2023-09-28 23:30:09,328 - utils - INFO - 1, epoch: 1276, all client loss: [0.525075376033783, 0.46564996242523193], all pred client disparities: [0.009901285171508789, 0.00036516785621643066], all client disparities: [0.005072444677352905, 0.002700597047805786], all client accs: [0.7457627654075623, 0.7873273491859436],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:09,565 - utils - INFO - stage2_gradient_single_runtime: 0.009603738784790039
2023-09-28 23:30:09,570 - utils - INFO - 1, epoch: 1277, all client loss: [0.5250179171562195, 0.46537280082702637], all pred client disparities: [0.009994357824325562, 0.0003401041030883789], all client disparities: [0.005072444677352905, 0.0026275068521499634], all client accs: [0.7457627654075623, 0.7872962355613708],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:09,828 - utils - INFO - stage2_gradient_single_runtime: 0.006239652633666992
2023-09-28 23:30:09,833 - utils - INFO - 1, epoch: 1278, all client loss: [0.5249602198600769, 0.46509861946105957], all pred client disparities: [0.010087549686431885, 0.0003171861171722412], all client disparities: [0.005072444677352905, 0.0025544166564941406], all client accs: [0.7457627654075623, 0.7874206900596619],  alphas:tensor([0.8770, 0.0000, 0.0000, 0.1230], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:10,068 - utils - INFO - stage2_gradient_single_runtime: 0.0063250064849853516
2023-09-28 23:30:10,073 - utils - INFO - 1, epoch: 1279, all client loss: [0.5249457359313965, 0.46483340859413147], all pred client disparities: [0.010167628526687622, 0.00020270049571990967], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.4803, 0.0000, 0.1200, 0.3997], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:10,203 - utils - INFO - valid: True, epoch: 1279, loss: [0.5794118642807007, 0.46446672081947327], accuracy: [0.7292817831039429, 0.7881987690925598], mean_accuracy:0.7587402760982513,variance_accuracy:0.02945849299430847, disparity: [0.009090900421142578, 0.006953105330467224], mean_disparity:0.008022002875804901,variance_disparity:0.001068897545337677, pred_disparity: [0.0002766549587249756, 0.0017608106136322021]
2023-09-28 23:30:10,286 - utils - INFO - global_valid: True, epoch: 1279,  global_loss: 0.46574464440345764, global_accuracy: 0.8145062280175164,  global_disparity:0.010382786393165588, global_pred_disparity: 0.005022779107093811,
2023-09-28 23:30:10,519 - utils - INFO - stage2_gradient_single_runtime: 0.006345510482788086
2023-09-28 23:30:10,524 - utils - INFO - 1, epoch: 1280, all client loss: [0.5249356627464294, 0.4649356007575989], all pred client disparities: [0.010121822357177734, 0.00018668174743652344], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.4802, 0.0000, 0.1201, 0.3997], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:10,760 - utils - INFO - stage2_gradient_single_runtime: 0.006261348724365234
2023-09-28 23:30:10,765 - utils - INFO - 1, epoch: 1281, all client loss: [0.5249262452125549, 0.4650372564792633], all pred client disparities: [0.01007649302482605, 0.0001697838306427002], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.8771, 0.0000, 0.0000, 0.1229], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:11,005 - utils - INFO - stage2_gradient_single_runtime: 0.006494045257568359
2023-09-28 23:30:11,010 - utils - INFO - 1, epoch: 1282, all client loss: [0.5249115824699402, 0.4647725522518158], all pred client disparities: [0.010156571865081787, 5.56558406969998e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.4804, 0.0000, 0.1202, 0.3994], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:11,251 - utils - INFO - stage2_gradient_single_runtime: 0.006695985794067383
2023-09-28 23:30:11,256 - utils - INFO - 1, epoch: 1283, all client loss: [0.5249013900756836, 0.46487462520599365], all pred client disparities: [0.010110616683959961, 3.9964914321899414e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.4803, 0.0000, 0.1203, 0.3993], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:11,508 - utils - INFO - stage2_gradient_single_runtime: 0.00627589225769043
2023-09-28 23:30:11,513 - utils - INFO - 1, epoch: 1284, all client loss: [0.5248918533325195, 0.4649761915206909], all pred client disparities: [0.010065227746963501, 2.3379927370115183e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([0.4783, 0.0000, 0.2201, 0.3016], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:11,756 - utils - INFO - stage2_gradient_single_runtime: 0.006362438201904297
2023-09-28 23:30:11,761 - utils - INFO - 1, epoch: 1285, all client loss: [0.5248218178749084, 0.46505042910575867], all pred client disparities: [0.009999394416809082, 0.00023216009140014648], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7878872752189636],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:11,997 - utils - INFO - stage2_gradient_single_runtime: 0.006282806396484375
2023-09-28 23:30:12,002 - utils - INFO - 1, epoch: 1286, all client loss: [0.5247656106948853, 0.4647771716117859], all pred client disparities: [0.010091990232467651, 0.00020740926265716553], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7880738973617554],  alphas:tensor([0.8797, 0.0000, 0.0000, 0.1203], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:12,238 - utils - INFO - stage2_gradient_single_runtime: 0.006320476531982422
2023-09-28 23:30:12,243 - utils - INFO - 1, epoch: 1287, all client loss: [0.5247506499290466, 0.4645130932331085], all pred client disparities: [0.010172396898269653, 9.390711784362793e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7881361246109009],  alphas:tensor([0.4806, 0.0000, 0.1211, 0.3983], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:12,480 - utils - INFO - stage2_gradient_single_runtime: 0.006254911422729492
2023-09-28 23:30:12,485 - utils - INFO - 1, epoch: 1288, all client loss: [0.52474045753479, 0.4646146297454834], all pred client disparities: [0.010126262903213501, 7.864832878112793e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7881050109863281],  alphas:tensor([0.4805, 0.0000, 0.1212, 0.3983], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:12,723 - utils - INFO - stage2_gradient_single_runtime: 0.00627589225769043
2023-09-28 23:30:12,728 - utils - INFO - 1, epoch: 1289, all client loss: [0.5247308015823364, 0.4647156596183777], all pred client disparities: [0.010080605745315552, 6.242096424102783e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7880738973617554],  alphas:tensor([0.4805, 0.0000, 0.1213, 0.3982], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:12,963 - utils - INFO - stage2_gradient_single_runtime: 0.006257295608520508
2023-09-28 23:30:12,968 - utils - INFO - 1, epoch: 1290, all client loss: [0.5247216820716858, 0.4648161232471466], all pred client disparities: [0.010035455226898193, 4.5344237150857225e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7880738973617554],  alphas:tensor([0.8831, 0.0000, 0.0000, 0.1169], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:13,202 - utils - INFO - stage2_gradient_single_runtime: 0.006269931793212891
2023-09-28 23:30:13,208 - utils - INFO - 1, epoch: 1291, all client loss: [0.5247059464454651, 0.4645507335662842], all pred client disparities: [0.010115951299667358, 6.7099928855896e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7881983518600464],  alphas:tensor([0.4900, 0.3327, 0.1773, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:13,443 - utils - INFO - stage2_gradient_single_runtime: 0.006288051605224609
2023-09-28 23:30:13,445 - utils - INFO - 1, epoch: 1292, all client loss: [0.5246720910072327, 0.46465492248535156], all pred client disparities: [0.010074913501739502, 2.7954578399658203e-05], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7881672382354736],  alphas:tensor([0.4890, 0.3332, 0.1779, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:13,684 - utils - INFO - stage2_gradient_single_runtime: 0.006240367889404297
2023-09-28 23:30:13,689 - utils - INFO - 1, epoch: 1293, all client loss: [0.5246395468711853, 0.4647577702999115], all pred client disparities: [0.010034620761871338, 8.940696716308594e-06], all client disparities: [0.005072444677352905, 0.002909436821937561], all client accs: [0.7457627654075623, 0.7880738973617554],  alphas:tensor([0.4788, 0.0000, 0.2178, 0.3034], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:13,923 - utils - INFO - stage2_gradient_single_runtime: 0.00723719596862793
2023-09-28 23:30:13,926 - utils - INFO - 1, epoch: 1294, all client loss: [0.5245696306228638, 0.46483317017555237], all pred client disparities: [0.009968161582946777, 0.0002195984125137329], all client disparities: [0.005072444677352905, 0.0030556172132492065], all client accs: [0.7457627654075623, 0.7881361246109009],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:14,160 - utils - INFO - stage2_gradient_single_runtime: 0.006373167037963867
2023-09-28 23:30:14,166 - utils - INFO - 1, epoch: 1295, all client loss: [0.5245159864425659, 0.4645584523677826], all pred client disparities: [0.010059922933578491, 0.00018960237503051758], all client disparities: [0.005072444677352905, 0.0030556172132492065], all client accs: [0.7457627654075623, 0.7881672382354736],  alphas:tensor([0.8885, 0.0000, 0.0000, 0.1115], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:14,406 - utils - INFO - stage2_gradient_single_runtime: 0.00632929801940918
2023-09-28 23:30:14,414 - utils - INFO - 1, epoch: 1296, all client loss: [0.5244996547698975, 0.46429261565208435], all pred client disparities: [0.010140806436538696, 7.814168930053711e-05], all client disparities: [0.005072444677352905, 0.0030556172132492065], all client accs: [0.7457627654075623, 0.7883227467536926],  alphas:tensor([0.4808, 0.0000, 0.1226, 0.3966], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:14,649 - utils - INFO - stage2_gradient_single_runtime: 0.00638270378112793
2023-09-28 23:30:14,651 - utils - INFO - 1, epoch: 1297, all client loss: [0.5244901180267334, 0.4643925726413727], all pred client disparities: [0.01009511947631836, 6.210803985595703e-05], all client disparities: [0.005072444677352905, 0.0030556172132492065], all client accs: [0.7457627654075623, 0.7882294058799744],  alphas:tensor([0.4807, 0.0000, 0.1227, 0.3966], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:14,891 - utils - INFO - stage2_gradient_single_runtime: 0.006249427795410156
2023-09-28 23:30:14,895 - utils - INFO - 1, epoch: 1298, all client loss: [0.5244811773300171, 0.4644920229911804], all pred client disparities: [0.01004984974861145, 4.51505184173584e-05], all client disparities: [0.005072444677352905, 0.0030556172132492065], all client accs: [0.7457627654075623, 0.7881983518600464],  alphas:tensor([0.4786, 0.0000, 0.2169, 0.3045], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:15,139 - utils - INFO - stage2_gradient_single_runtime: 0.0062673091888427734
2023-09-28 23:30:15,144 - utils - INFO - 1, epoch: 1299, all client loss: [0.5244109630584717, 0.4645686149597168], all pred client disparities: [0.009982556104660034, 0.00025747716426849365], all client disparities: [0.005072444677352905, 0.0030556172132492065], all client accs: [0.7457627654075623, 0.7881361246109009],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:15,227 - utils - INFO - valid: True, epoch: 1299, loss: [0.5792508721351624, 0.4638095796108246], accuracy: [0.7292817831039429, 0.7884472012519836], mean_accuracy:0.7588644921779633,variance_accuracy:0.029582709074020386, disparity: [0.009090900421142578, 0.006953105330467224], mean_disparity:0.008022002875804901,variance_disparity:0.001068897545337677, pred_disparity: [0.0010283887386322021, 0.0015650838613510132]
2023-09-28 23:30:15,353 - utils - INFO - global_valid: True, epoch: 1299,  global_loss: 0.4650929868221283, global_accuracy: 0.8150914657414605,  global_disparity:0.010382786393165588, global_pred_disparity: 0.004859760403633118,
2023-09-28 23:30:15,587 - utils - INFO - stage2_gradient_single_runtime: 0.0062181949615478516
2023-09-28 23:30:15,592 - utils - INFO - 1, epoch: 1300, all client loss: [0.5243584513664246, 0.46429458260536194], all pred client disparities: [0.010073840618133545, 0.0002257823944091797], all client disparities: [0.005072444677352905, 0.0030556172132492065], all client accs: [0.7457627654075623, 0.7882605195045471],  alphas:tensor([0.4807, 0.0000, 0.1235, 0.3958], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:15,824 - utils - INFO - stage2_gradient_single_runtime: 0.006228446960449219
2023-09-28 23:30:15,829 - utils - INFO - 1, epoch: 1301, all client loss: [0.5243500471115112, 0.46439307928085327], all pred client disparities: [0.010028928518295288, 0.00020802021026611328], all client disparities: [0.005072444677352905, 0.0030556172132492065], all client accs: [0.7457627654075623, 0.7881983518600464],  alphas:tensor([0.8942, 0.0000, 0.0000, 0.1058], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:16,072 - utils - INFO - stage2_gradient_single_runtime: 0.006422996520996094
2023-09-28 23:30:16,077 - utils - INFO - 1, epoch: 1302, all client loss: [0.5243328213691711, 0.46412625908851624], all pred client disparities: [0.010110408067703247, 9.784102439880371e-05], all client disparities: [0.005072444677352905, 0.002836361527442932], all client accs: [0.7457627654075623, 0.7882916331291199],  alphas:tensor([0.4809, 0.0000, 0.1236, 0.3955], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:16,311 - utils - INFO - stage2_gradient_single_runtime: 0.006270885467529297
2023-09-28 23:30:16,313 - utils - INFO - 1, epoch: 1303, all client loss: [0.5243237614631653, 0.4642251431941986], all pred client disparities: [0.010064929723739624, 8.127093315124512e-05], all client disparities: [0.005072444677352905, 0.002836361527442932], all client accs: [0.7457627654075623, 0.7882916331291199],  alphas:tensor([0.4808, 0.0000, 0.1237, 0.3955], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:16,552 - utils - INFO - stage2_gradient_single_runtime: 0.006263017654418945
2023-09-28 23:30:16,555 - utils - INFO - 1, epoch: 1304, all client loss: [0.5243152976036072, 0.4643234610557556], all pred client disparities: [0.010019958019256592, 6.374716758728027e-05], all client disparities: [0.005072444677352905, 0.002836361527442932], all client accs: [0.7457627654075623, 0.7882294058799744],  alphas:tensor([0.8940, 0.0000, 0.0000, 0.1060], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:16,791 - utils - INFO - stage2_gradient_single_runtime: 0.006230831146240234
2023-09-28 23:30:16,796 - utils - INFO - 1, epoch: 1305, all client loss: [0.5242980122566223, 0.46405720710754395], all pred client disparities: [0.010101288557052612, 4.620850450010039e-05], all client disparities: [0.005072444677352905, 0.0027632713317871094], all client accs: [0.7457627654075623, 0.7882605195045471],  alphas:tensor([0.4925, 0.3269, 0.1806, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:17,031 - utils - INFO - stage2_gradient_single_runtime: 0.006291389465332031
2023-09-28 23:30:17,036 - utils - INFO - 1, epoch: 1306, all client loss: [0.524264395236969, 0.4641604721546173], all pred client disparities: [0.010059744119644165, 4.366065695649013e-06], all client disparities: [0.005072444677352905, 0.002836361527442932], all client accs: [0.7457627654075623, 0.7882916331291199],  alphas:tensor([0.4913, 0.3275, 0.1812, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:17,275 - utils - INFO - stage2_gradient_single_runtime: 0.006238698959350586
2023-09-28 23:30:17,280 - utils - INFO - 1, epoch: 1307, all client loss: [0.5242319107055664, 0.4642622768878937], all pred client disparities: [0.010018855333328247, 3.509223824949004e-05], all client disparities: [0.005072444677352905, 0.002836361527442932], all client accs: [0.7457627654075623, 0.7882916331291199],  alphas:tensor([0.8972, 0.0000, 0.0000, 0.1028], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:17,518 - utils - INFO - stage2_gradient_single_runtime: 0.0063018798828125
2023-09-28 23:30:17,522 - utils - INFO - 1, epoch: 1308, all client loss: [0.5242142081260681, 0.4639953076839447], all pred client disparities: [0.010100454092025757, 7.411837577819824e-05], all client disparities: [0.00688403844833374, 0.0027632713317871094], all client accs: [0.7433414459228516, 0.7882605195045471],  alphas:tensor([0.4929, 0.3258, 0.1813, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:17,759 - utils - INFO - stage2_gradient_single_runtime: 0.006246805191040039
2023-09-28 23:30:17,764 - utils - INFO - 1, epoch: 1309, all client loss: [0.5241808295249939, 0.46409812569618225], all pred client disparities: [0.010058790445327759, 3.21716106554959e-05], all client disparities: [0.005072444677352905, 0.0027632713317871094], all client accs: [0.7457627654075623, 0.7882605195045471],  alphas:tensor([0.4917, 0.3265, 0.1819, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:18,000 - utils - INFO - stage2_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:30:18,005 - utils - INFO - 1, epoch: 1310, all client loss: [0.5241486430168152, 0.4641995131969452], all pred client disparities: [0.01001805067062378, 7.346287475229474e-06], all client disparities: [0.005072444677352905, 0.0027632713317871094], all client accs: [0.7457627654075623, 0.7882605195045471],  alphas:tensor([0.4795, 0.0000, 0.2144, 0.3061], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:18,242 - utils - INFO - stage2_gradient_single_runtime: 0.0062923431396484375
2023-09-28 23:30:18,247 - utils - INFO - 1, epoch: 1311, all client loss: [0.5240787863731384, 0.46427759528160095], all pred client disparities: [0.009949684143066406, 0.00022248923778533936], all client disparities: [0.005072444677352905, 0.002836361527442932], all client accs: [0.7457627654075623, 0.7882294058799744],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:18,484 - utils - INFO - stage2_gradient_single_runtime: 0.0063228607177734375
2023-09-28 23:30:18,489 - utils - INFO - 1, epoch: 1312, all client loss: [0.5240294933319092, 0.46400168538093567], all pred client disparities: [0.010040223598480225, 0.00018388032913208008], all client disparities: [0.00688403844833374, 0.0027632713317871094], all client accs: [0.7433414459228516, 0.7881983518600464],  alphas:tensor([0.4809, 0.0000, 0.1255, 0.3936], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:18,676 - utils - INFO - stage2_gradient_single_runtime: 0.006289243698120117
2023-09-28 23:30:18,681 - utils - INFO - 1, epoch: 1313, all client loss: [0.5240219831466675, 0.46409791707992554], all pred client disparities: [0.009995818138122559, 0.00016488134860992432], all client disparities: [0.005072444677352905, 0.0027632713317871094], all client accs: [0.7457627654075623, 0.7881983518600464],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:18,914 - utils - INFO - stage2_gradient_single_runtime: 0.006246805191040039
2023-09-28 23:30:18,919 - utils - INFO - 1, epoch: 1314, all client loss: [0.5239728093147278, 0.46382367610931396], all pred client disparities: [0.010086297988891602, 0.00012715160846710205], all client disparities: [0.00688403844833374, 0.0027632713317871094], all client accs: [0.7433414459228516, 0.7882605195045471],  alphas:tensor([0.4810, 0.0000, 0.1259, 0.3931], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:19,154 - utils - INFO - stage2_gradient_single_runtime: 0.006231784820556641
2023-09-28 23:30:19,159 - utils - INFO - 1, epoch: 1315, all client loss: [0.5239648818969727, 0.46392005681991577], all pred client disparities: [0.010041415691375732, 0.00010904669761657715], all client disparities: [0.00688403844833374, 0.0027632713317871094], all client accs: [0.7433414459228516, 0.7882605195045471],  alphas:tensor([0.4810, 0.0000, 0.1259, 0.3931], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:19,393 - utils - INFO - stage2_gradient_single_runtime: 0.006315469741821289
2023-09-28 23:30:19,398 - utils - INFO - 1, epoch: 1316, all client loss: [0.5239575505256653, 0.4640159606933594], all pred client disparities: [0.009997069835662842, 9.001791477203369e-05], all client disparities: [0.00688403844833374, 0.0027632713317871094], all client accs: [0.7433414459228516, 0.7881983518600464],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:19,632 - utils - INFO - stage2_gradient_single_runtime: 0.006426572799682617
2023-09-28 23:30:19,637 - utils - INFO - 1, epoch: 1317, all client loss: [0.523908793926239, 0.4637417495250702], all pred client disparities: [0.010087430477142334, 5.148351556272246e-05], all client disparities: [0.00688403844833374, 0.001959294080734253], all client accs: [0.7433414459228516, 0.7881672382354736],  alphas:tensor([0.4811, 0.0000, 0.1263, 0.3926], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:19,873 - utils - INFO - stage2_gradient_single_runtime: 0.006265163421630859
2023-09-28 23:30:19,878 - utils - INFO - 1, epoch: 1318, all client loss: [0.5239009857177734, 0.4638378322124481], all pred client disparities: [0.010042577981948853, 3.336370355100371e-05], all client disparities: [0.00688403844833374, 0.0027632713317871094], all client accs: [0.7433414459228516, 0.7882294058799744],  alphas:tensor([0.4810, 0.0000, 0.1264, 0.3926], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:20,112 - utils - INFO - stage2_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:30:20,117 - utils - INFO - 1, epoch: 1319, all client loss: [0.5238936543464661, 0.46393337845802307], all pred client disparities: [0.009998202323913574, 1.430511474609375e-05], all client disparities: [0.00688403844833374, 0.0027632713317871094], all client accs: [0.7433414459228516, 0.7882294058799744],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:20,258 - utils - INFO - valid: True, epoch: 1319, loss: [0.5791129469871521, 0.4631507992744446], accuracy: [0.7292817831039429, 0.7885093092918396], mean_accuracy:0.7588955461978912,variance_accuracy:0.029613763093948364, disparity: [0.009090900421142578, 0.007396891713142395], mean_disparity:0.008243896067142487,variance_disparity:0.0008470043540000916, pred_disparity: [0.0016975700855255127, 0.001635625958442688]
2023-09-28 23:30:20,337 - utils - INFO - global_valid: True, epoch: 1319,  global_loss: 0.4644400179386139, global_accuracy: 0.8158798189482868,  global_disparity:0.01081259548664093, global_pred_disparity: 0.004953354597091675,
2023-09-28 23:30:20,572 - utils - INFO - stage2_gradient_single_runtime: 0.006313323974609375
2023-09-28 23:30:20,578 - utils - INFO - 1, epoch: 1320, all client loss: [0.523845374584198, 0.46365925669670105], all pred client disparities: [0.010088354349136353, 2.5048855604836717e-05], all client disparities: [0.00688403844833374, 0.0017400234937667847], all client accs: [0.7433414459228516, 0.7880427837371826],  alphas:tensor([0.4938, 0.3215, 0.1847, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:20,812 - utils - INFO - stage2_gradient_single_runtime: 0.0062901973724365234
2023-09-28 23:30:20,818 - utils - INFO - 1, epoch: 1321, all client loss: [0.5238131284713745, 0.4637600779533386], all pred client disparities: [0.010046899318695068, 1.706183684291318e-05], all client disparities: [0.00688403844833374, 0.0017400234937667847], all client accs: [0.7433414459228516, 0.7880427837371826],  alphas:tensor([0.4810, 0.0000, 0.1269, 0.3920], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:21,052 - utils - INFO - stage2_gradient_single_runtime: 0.0062906742095947266
2023-09-28 23:30:21,057 - utils - INFO - 1, epoch: 1322, all client loss: [0.523806095123291, 0.4638550281524658], all pred client disparities: [0.01000264286994934, 2.294778823852539e-06], all client disparities: [0.00688403844833374, 0.003139123320579529], all client accs: [0.7433414459228516, 0.7884782552719116],  alphas:tensor([0.5313, 0.0000, 0.2105, 0.2582], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:21,293 - utils - INFO - stage2_gradient_single_runtime: 0.006259441375732422
2023-09-28 23:30:21,298 - utils - INFO - 1, epoch: 1323, all client loss: [0.5238617658615112, 0.46379148960113525], all pred client disparities: [0.0099753737449646, 0.00017407536506652832], all client disparities: [0.00688403844833374, 0.0017400234937667847], all client accs: [0.7433414459228516, 0.7880427837371826],  alphas:tensor([0.8481, 0.0000, 0.1519, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:21,529 - utils - INFO - stage2_gradient_single_runtime: 0.006274223327636719
2023-09-28 23:30:21,534 - utils - INFO - 1, epoch: 1324, all client loss: [0.5236702561378479, 0.4641295075416565], all pred client disparities: [0.009889930486679077, 0.000332564115524292], all client disparities: [0.005072444677352905, 0.00030940771102905273], all client accs: [0.7457627654075623, 0.7890381813049316],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:21,770 - utils - INFO - stage2_gradient_single_runtime: 0.006404399871826172
2023-09-28 23:30:21,775 - utils - INFO - 1, epoch: 1325, all client loss: [0.5236254930496216, 0.46384841203689575], all pred client disparities: [0.009979337453842163, 0.00028210878372192383], all client disparities: [0.00688403844833374, 0.003671616315841675], all client accs: [0.7433414459228516, 0.7889137864112854],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:22,006 - utils - INFO - stage2_gradient_single_runtime: 0.006295919418334961
2023-09-28 23:30:22,011 - utils - INFO - 1, epoch: 1326, all client loss: [0.5235802531242371, 0.46357059478759766], all pred client disparities: [0.010069042444229126, 0.00023448467254638672], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7887582182884216],  alphas:tensor([0.4810, 0.0000, 0.1284, 0.3906], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:22,244 - utils - INFO - stage2_gradient_single_runtime: 0.0062754154205322266
2023-09-28 23:30:22,249 - utils - INFO - 1, epoch: 1327, all client loss: [0.5235739350318909, 0.46366408467292786], all pred client disparities: [0.01002514362335205, 0.00021404027938842773], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7887582182884216],  alphas:tensor([0.9212, 0.0000, 0.0000, 0.0788], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:22,484 - utils - INFO - stage2_gradient_single_runtime: 0.006221771240234375
2023-09-28 23:30:22,489 - utils - INFO - 1, epoch: 1328, all client loss: [0.5235530734062195, 0.4633925259113312], all pred client disparities: [0.010108500719070435, 0.00010967254638671875], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([0.4811, 0.0000, 0.1286, 0.3903], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:22,722 - utils - INFO - stage2_gradient_single_runtime: 0.0062906742095947266
2023-09-28 23:30:22,726 - utils - INFO - 1, epoch: 1329, all client loss: [0.5235461592674255, 0.4634864032268524], all pred client disparities: [0.010064095258712769, 9.043514728546143e-05], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([0.4811, 0.0000, 0.1287, 0.3903], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:22,962 - utils - INFO - stage2_gradient_single_runtime: 0.006249427795410156
2023-09-28 23:30:22,967 - utils - INFO - 1, epoch: 1330, all client loss: [0.5235397219657898, 0.4635797441005707], all pred client disparities: [0.010020047426223755, 7.0229172706604e-05], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([0.9205, 0.0000, 0.0000, 0.0795], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:23,203 - utils - INFO - stage2_gradient_single_runtime: 0.0062296390533447266
2023-09-28 23:30:23,208 - utils - INFO - 1, epoch: 1331, all client loss: [0.523518979549408, 0.46330899000167847], all pred client disparities: [0.010103434324264526, 3.410876161069609e-05], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([0.4950, 0.3173, 0.1877, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:23,442 - utils - INFO - stage2_gradient_single_runtime: 0.006331205368041992
2023-09-28 23:30:23,447 - utils - INFO - 1, epoch: 1332, all client loss: [0.523487389087677, 0.46340838074684143], all pred client disparities: [0.010061800479888916, 8.896005965652876e-06], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([0.4811, 0.0000, 0.1291, 0.3898], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:23,681 - utils - INFO - stage2_gradient_single_runtime: 0.0062673091888427734
2023-09-28 23:30:23,686 - utils - INFO - 1, epoch: 1333, all client loss: [0.5234810709953308, 0.4635013937950134], all pred client disparities: [0.010017961263656616, 1.1369595995347481e-05], all client disparities: [0.00688403844833374, 0.002721458673477173], all client accs: [0.7433414459228516, 0.7889759540557861],  alphas:tensor([0.5331, 0.0000, 0.2081, 0.2589], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:23,922 - utils - INFO - stage2_gradient_single_runtime: 0.00643467903137207
2023-09-28 23:30:23,927 - utils - INFO - 1, epoch: 1334, all client loss: [0.5235350728034973, 0.4634389281272888], all pred client disparities: [0.009990990161895752, 0.0001827329397201538], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([0.8377, 0.0000, 0.1623, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:24,161 - utils - INFO - stage2_gradient_single_runtime: 0.0062634944915771484
2023-09-28 23:30:24,166 - utils - INFO - 1, epoch: 1335, all client loss: [0.5233519077301025, 0.46376609802246094], all pred client disparities: [0.009905368089675903, 0.0003130584955215454], all client disparities: [0.00688403844833374, 0.0007138252258300781], all client accs: [0.7433414459228516, 0.7888204455375671],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:24,401 - utils - INFO - stage2_gradient_single_runtime: 0.006257057189941406
2023-09-28 23:30:24,406 - utils - INFO - 1, epoch: 1336, all client loss: [0.523309588432312, 0.46348443627357483], all pred client disparities: [0.00999438762664795, 0.00025707483291625977], all client disparities: [0.00688403844833374, 0.0007138252258300781], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:24,639 - utils - INFO - stage2_gradient_single_runtime: 0.006247043609619141
2023-09-28 23:30:24,645 - utils - INFO - 1, epoch: 1337, all client loss: [0.5232669115066528, 0.4632061719894409], all pred client disparities: [0.010083824396133423, 0.00020410120487213135], all client disparities: [0.00688403844833374, 0.0007138252258300781], all client accs: [0.7433414459228516, 0.7891315221786499],  alphas:tensor([0.4810, 0.0000, 0.1306, 0.3884], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:24,879 - utils - INFO - stage2_gradient_single_runtime: 0.0062103271484375
2023-09-28 23:30:24,884 - utils - INFO - 1, epoch: 1338, all client loss: [0.5232611298561096, 0.4632977843284607], all pred client disparities: [0.010040253400802612, 0.00018295645713806152], all client disparities: [0.00688403844833374, 0.0007138252258300781], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.9302, 0.0000, 0.0000, 0.0698], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:25,118 - utils - INFO - stage2_gradient_single_runtime: 0.006268978118896484
2023-09-28 23:30:25,123 - utils - INFO - 1, epoch: 1339, all client loss: [0.5232393145561218, 0.4630255103111267], all pred client disparities: [0.010124385356903076, 8.04513692855835e-05], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([0.4811, 0.0000, 0.1308, 0.3881], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:25,207 - utils - INFO - valid: True, epoch: 1339, loss: [0.5789742469787598, 0.46259093284606934], accuracy: [0.7292817831039429, 0.7893167734146118], mean_accuracy:0.7592992782592773,variance_accuracy:0.030017495155334473, disparity: [0.009090900421142578, 0.008258432149887085], mean_disparity:0.008674666285514832,variance_disparity:0.0004162341356277466, pred_disparity: [0.002253323793411255, 0.0013936161994934082]
2023-09-28 23:30:25,339 - utils - INFO - global_valid: True, epoch: 1339,  global_loss: 0.4638848304748535, global_accuracy: 0.816506750338369,  global_disparity:0.011615842580795288, global_pred_disparity: 0.004733622074127197,
2023-09-28 23:30:25,577 - utils - INFO - stage2_gradient_single_runtime: 0.006319284439086914
2023-09-28 23:30:25,582 - utils - INFO - 1, epoch: 1340, all client loss: [0.5232329964637756, 0.46311748027801514], all pred client disparities: [0.010080188512802124, 6.0483816923806444e-05], all client disparities: [0.00688403844833374, 0.00264836847782135], all client accs: [0.7433414459228516, 0.7890070676803589],  alphas:tensor([0.4811, 0.0000, 0.1308, 0.3881], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:25,817 - utils - INFO - stage2_gradient_single_runtime: 0.006279706954956055
2023-09-28 23:30:25,822 - utils - INFO - 1, epoch: 1341, all client loss: [0.5232270956039429, 0.46320900321006775], all pred client disparities: [0.010036438703536987, 3.959238893003203e-05], all client disparities: [0.00688403844833374, 0.00078691536327824], all client accs: [0.7433414459228516, 0.7891315221786499],  alphas:tensor([0.4810, 0.0000, 0.1309, 0.3881], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:26,054 - utils - INFO - stage2_gradient_single_runtime: 0.006356954574584961
2023-09-28 23:30:26,059 - utils - INFO - 1, epoch: 1342, all client loss: [0.5232217311859131, 0.46330001950263977], all pred client disparities: [0.009993135929107666, 1.774728843884077e-05], all client disparities: [0.00688403844833374, 0.00078691536327824], all client accs: [0.7433414459228516, 0.7891315221786499],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:26,293 - utils - INFO - stage2_gradient_single_runtime: 0.006277322769165039
2023-09-28 23:30:26,298 - utils - INFO - 1, epoch: 1343, all client loss: [0.5231793522834778, 0.46302300691604614], all pred client disparities: [0.010082513093948364, 3.49581241607666e-05], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7891315221786499],  alphas:tensor([0.4949, 0.3139, 0.1912, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:26,528 - utils - INFO - stage2_gradient_single_runtime: 0.006302833557128906
2023-09-28 23:30:26,533 - utils - INFO - 1, epoch: 1344, all client loss: [0.5231492519378662, 0.463119775056839], all pred client disparities: [0.010041475296020508, 6.75024239171762e-06], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.4810, 0.0000, 0.1314, 0.3875], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:26,767 - utils - INFO - stage2_gradient_single_runtime: 0.0062253475189208984
2023-09-28 23:30:26,773 - utils - INFO - 1, epoch: 1345, all client loss: [0.523144006729126, 0.4632103443145752], all pred client disparities: [0.009998321533203125, 1.5273697499651462e-05], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:27,007 - utils - INFO - stage2_gradient_single_runtime: 0.006285905838012695
2023-09-28 23:30:27,012 - utils - INFO - 1, epoch: 1346, all client loss: [0.5231022238731384, 0.4629332721233368], all pred client disparities: [0.010087579488754272, 6.923079490661621e-05], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.4952, 0.3129, 0.1920, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:27,248 - utils - INFO - stage2_gradient_single_runtime: 0.006316184997558594
2023-09-28 23:30:27,253 - utils - INFO - 1, epoch: 1347, all client loss: [0.5230722427368164, 0.4630297124385834], all pred client disparities: [0.010046511888504028, 2.7298927307128906e-05], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.4939, 0.3137, 0.1924, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:27,487 - utils - INFO - stage2_gradient_single_runtime: 0.006219148635864258
2023-09-28 23:30:27,492 - utils - INFO - 1, epoch: 1348, all client loss: [0.5230435132980347, 0.46312475204467773], all pred client disparities: [0.01000627875328064, 1.2069940567016602e-05], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7890692949295044],  alphas:tensor([0.9386, 0.0000, 0.0000, 0.0614], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:27,725 - utils - INFO - stage2_gradient_single_runtime: 0.006287574768066406
2023-09-28 23:30:27,731 - utils - INFO - 1, epoch: 1349, all client loss: [0.5230206251144409, 0.46285074949264526], all pred client disparities: [0.010090917348861694, 8.855760097503662e-05], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.4953, 0.3119, 0.1928, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:27,973 - utils - INFO - stage2_gradient_single_runtime: 0.006342649459838867
2023-09-28 23:30:27,979 - utils - INFO - 1, epoch: 1350, all client loss: [0.5229910016059875, 0.46294674277305603], all pred client disparities: [0.010049939155578613, 4.665553933591582e-05], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.4940, 0.3127, 0.1933, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:28,224 - utils - INFO - stage2_gradient_single_runtime: 0.006404399871826172
2023-09-28 23:30:28,231 - utils - INFO - 1, epoch: 1351, all client loss: [0.5229624509811401, 0.4630412459373474], all pred client disparities: [0.010009765625, 7.361173629760742e-06], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7890692949295044],  alphas:tensor([0.5353, 0.0000, 0.2029, 0.2619], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:28,467 - utils - INFO - stage2_gradient_single_runtime: 0.006273746490478516
2023-09-28 23:30:28,472 - utils - INFO - 1, epoch: 1352, all client loss: [0.5230140089988708, 0.4629804790019989], all pred client disparities: [0.00998300313949585, 0.00017750263214111328], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.8195, 0.0000, 0.1805, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:28,709 - utils - INFO - stage2_gradient_single_runtime: 0.0062732696533203125
2023-09-28 23:30:28,716 - utils - INFO - 1, epoch: 1353, all client loss: [0.5228442549705505, 0.46328991651535034], all pred client disparities: [0.009899109601974487, 0.00029981136322021484], all client disparities: [0.00688403844833374, 0.0015073716640472412], all client accs: [0.7433414459228516, 0.7890381813049316],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:28,953 - utils - INFO - stage2_gradient_single_runtime: 0.0062444210052490234
2023-09-28 23:30:28,959 - utils - INFO - 1, epoch: 1354, all client loss: [0.5228060483932495, 0.46300598978996277], all pred client disparities: [0.009987860918045044, 0.0002334117889404297], all client disparities: [0.00688403844833374, 0.0008600056171417236], all client accs: [0.7433414459228516, 0.7889759540557861],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:29,196 - utils - INFO - stage2_gradient_single_runtime: 0.006258249282836914
2023-09-28 23:30:29,203 - utils - INFO - 1, epoch: 1355, all client loss: [0.5227674841880798, 0.46272557973861694], all pred client disparities: [0.010076940059661865, 0.0001703500747680664], all client disparities: [0.00688403844833374, 0.0013611912727355957], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4809, 0.0000, 0.1341, 0.3849], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:29,440 - utils - INFO - stage2_gradient_single_runtime: 0.006235837936401367
2023-09-28 23:30:29,447 - utils - INFO - 1, epoch: 1356, all client loss: [0.5227630138397217, 0.4628138244152069], all pred client disparities: [0.010034352540969849, 0.00014708936214447021], all client disparities: [0.00688403844833374, 0.0013611912727355957], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.9478, 0.0000, 0.0000, 0.0522], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:29,683 - utils - INFO - stage2_gradient_single_runtime: 0.006260395050048828
2023-09-28 23:30:29,690 - utils - INFO - 1, epoch: 1357, all client loss: [0.5227392911911011, 0.46253859996795654], all pred client disparities: [0.010119795799255371, 4.8145655455300584e-05], all client disparities: [0.00688403844833374, 0.0013611912727355957], all client accs: [0.7433414459228516, 0.7892248034477234],  alphas:tensor([0.4810, 0.0000, 0.1344, 0.3846], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:29,929 - utils - INFO - stage2_gradient_single_runtime: 0.0062694549560546875
2023-09-28 23:30:29,936 - utils - INFO - 1, epoch: 1358, all client loss: [0.5227342844009399, 0.46262726187705994], all pred client disparities: [0.010076522827148438, 2.6091936888406053e-05], all client disparities: [0.00688403844833374, 0.0013611912727355957], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4809, 0.0000, 0.1344, 0.3846], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:30,174 - utils - INFO - stage2_gradient_single_runtime: 0.006234645843505859
2023-09-28 23:30:30,182 - utils - INFO - 1, epoch: 1359, all client loss: [0.522729754447937, 0.46271538734436035], all pred client disparities: [0.01003369688987732, 3.0994415283203125e-06], all client disparities: [0.00688403844833374, 0.0013611912727355957], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4809, 0.0000, 0.1344, 0.3847], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:30,266 - utils - INFO - valid: True, epoch: 1359, loss: [0.5789405703544617, 0.4622630178928375], accuracy: [0.7292817831039429, 0.7897515296936035], mean_accuracy:0.7595166563987732,variance_accuracy:0.030234873294830322, disparity: [0.009090900421142578, 0.005891561508178711], mean_disparity:0.0074912309646606445,variance_disparity:0.0015996694564819336, pred_disparity: [0.0027135908603668213, 0.0013209134340286255]
2023-09-28 23:30:30,393 - utils - INFO - global_valid: True, epoch: 1359,  global_loss: 0.46356016397476196, global_accuracy: 0.8169248788447718,  global_disparity:0.00932358205318451, global_pred_disparity: 0.0046761780977249146,
2023-09-28 23:30:30,628 - utils - INFO - stage2_gradient_single_runtime: 0.0062961578369140625
2023-09-28 23:30:30,633 - utils - INFO - 1, epoch: 1360, all client loss: [0.5227257013320923, 0.46280306577682495], all pred client disparities: [0.009991228580474854, 2.078712532238569e-05], all client disparities: [0.00688403844833374, 0.0013611912727355957], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:30,872 - utils - INFO - stage2_gradient_single_runtime: 0.006232738494873047
2023-09-28 23:30:30,877 - utils - INFO - 1, epoch: 1361, all client loss: [0.5226871967315674, 0.46252432465553284], all pred client disparities: [0.010080456733703613, 8.296966552734375e-05], all client disparities: [0.00688403844833374, 0.0013611912727355957], all client accs: [0.7433414459228516, 0.7892248034477234],  alphas:tensor([0.8119, 0.0000, 0.1881, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:31,112 - utils - INFO - stage2_gradient_single_runtime: 0.006265401840209961
2023-09-28 23:30:31,117 - utils - INFO - 1, epoch: 1362, all client loss: [0.5225232243537903, 0.46282532811164856], all pred client disparities: [0.00999554991722107, 0.0003855973482131958], all client disparities: [0.00688403844833374, 0.002081647515296936], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:31,361 - utils - INFO - stage2_gradient_single_runtime: 0.006934404373168945
2023-09-28 23:30:31,366 - utils - INFO - 1, epoch: 1363, all client loss: [0.5224871635437012, 0.46254244446754456], all pred client disparities: [0.010084539651870728, 0.00031523406505584717], all client disparities: [0.00688403844833374, 0.002081647515296936], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.9579, 0.0000, 0.0000, 0.0421], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:31,555 - utils - INFO - stage2_gradient_single_runtime: 0.00629878044128418
2023-09-28 23:30:31,560 - utils - INFO - 1, epoch: 1364, all client loss: [0.5224624872207642, 0.46226561069488525], all pred client disparities: [0.010170996189117432, 0.000218123197555542], all client disparities: [0.00688403844833374, 0.0013611912727355957], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([0.4809, 0.0000, 0.1364, 0.3827], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:31,792 - utils - INFO - stage2_gradient_single_runtime: 0.006284475326538086
2023-09-28 23:30:31,797 - utils - INFO - 1, epoch: 1365, all client loss: [0.5224580764770508, 0.4623526334762573], all pred client disparities: [0.010128051042556763, 0.0001952648162841797], all client disparities: [0.00688403844833374, 0.002081647515296936], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4808, 0.0000, 0.1364, 0.3827], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:32,030 - utils - INFO - stage2_gradient_single_runtime: 0.006293535232543945
2023-09-28 23:30:32,035 - utils - INFO - 1, epoch: 1366, all client loss: [0.522454023361206, 0.4624392092227936], all pred client disparities: [0.010085493326187134, 0.00017152726650238037], all client disparities: [0.00688403844833374, 0.002081647515296936], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([0.4808, 0.0000, 0.1364, 0.3827], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:32,267 - utils - INFO - stage2_gradient_single_runtime: 0.006216287612915039
2023-09-28 23:30:32,272 - utils - INFO - 1, epoch: 1367, all client loss: [0.5224505066871643, 0.46252530813217163], all pred client disparities: [0.010043472051620483, 0.00014691054821014404], all client disparities: [0.00688403844833374, 0.002081647515296936], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([0.9594, 0.0000, 0.0000, 0.0406], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:32,504 - utils - INFO - stage2_gradient_single_runtime: 0.006345033645629883
2023-09-28 23:30:32,509 - utils - INFO - 1, epoch: 1368, all client loss: [0.5224254727363586, 0.4622481167316437], all pred client disparities: [0.010129868984222412, 5.02467155456543e-05], all client disparities: [0.00688403844833374, 0.002081647515296936], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4808, 0.0000, 0.1367, 0.3824], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:32,742 - utils - INFO - stage2_gradient_single_runtime: 0.00622868537902832
2023-09-28 23:30:32,747 - utils - INFO - 1, epoch: 1369, all client loss: [0.5224213600158691, 0.46233460307121277], all pred client disparities: [0.01008722186088562, 2.682209014892578e-05], all client disparities: [0.00688403844833374, 0.002081647515296936], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4808, 0.0000, 0.1367, 0.3825], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:32,979 - utils - INFO - stage2_gradient_single_runtime: 0.006398916244506836
2023-09-28 23:30:32,985 - utils - INFO - 1, epoch: 1370, all client loss: [0.5224176049232483, 0.46242067217826843], all pred client disparities: [0.01004493236541748, 2.4885384846129455e-06], all client disparities: [0.00688403844833374, 0.002081647515296936], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4847, 0.0000, 0.2082, 0.3071], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:33,219 - utils - INFO - stage2_gradient_single_runtime: 0.006309986114501953
2023-09-28 23:30:33,224 - utils - INFO - 1, epoch: 1371, all client loss: [0.5223495364189148, 0.46250611543655396], all pred client disparities: [0.009970128536224365, 0.00023451447486877441], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:33,452 - utils - INFO - stage2_gradient_single_runtime: 0.006305217742919922
2023-09-28 23:30:33,455 - utils - INFO - 1, epoch: 1372, all client loss: [0.522314190864563, 0.4622245728969574], all pred client disparities: [0.010059326887130737, 0.00016297399997711182], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([0.4807, 0.0000, 0.1376, 0.3817], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:33,695 - utils - INFO - stage2_gradient_single_runtime: 0.006261587142944336
2023-09-28 23:30:33,700 - utils - INFO - 1, epoch: 1373, all client loss: [0.5223107933998108, 0.4623098075389862], all pred client disparities: [0.010017454624176025, 0.00013807415962219238], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([0.4807, 0.0000, 0.1375, 0.3817], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:33,930 - utils - INFO - stage2_gradient_single_runtime: 0.006254434585571289
2023-09-28 23:30:33,935 - utils - INFO - 1, epoch: 1374, all client loss: [0.522307813167572, 0.4623946249485016], all pred client disparities: [0.009976029396057129, 0.00011232495307922363], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:34,168 - utils - INFO - stage2_gradient_single_runtime: 0.00629425048828125
2023-09-28 23:30:34,173 - utils - INFO - 1, epoch: 1375, all client loss: [0.5222725868225098, 0.46211403608322144], all pred client disparities: [0.010065257549285889, 4.138052827329375e-05], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.4807, 0.0000, 0.1379, 0.3814], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:34,406 - utils - INFO - stage2_gradient_single_runtime: 0.006268978118896484
2023-09-28 23:30:34,411 - utils - INFO - 1, epoch: 1376, all client loss: [0.522269070148468, 0.4621991813182831], all pred client disparities: [0.010023295879364014, 1.6793615941423923e-05], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.4807, 0.0000, 0.1379, 0.3814], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:34,643 - utils - INFO - stage2_gradient_single_runtime: 0.0062160491943359375
2023-09-28 23:30:34,648 - utils - INFO - 1, epoch: 1377, all client loss: [0.5222659707069397, 0.4622838795185089], all pred client disparities: [0.009981781244277954, 8.702278137207031e-06], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:34,881 - utils - INFO - stage2_gradient_single_runtime: 0.00629734992980957
2023-09-28 23:30:34,886 - utils - INFO - 1, epoch: 1378, all client loss: [0.5222307443618774, 0.46200427412986755], all pred client disparities: [0.010071098804473877, 7.905066013336182e-05], all client disparities: [0.00688403844833374, 0.0030213892459869385], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([0.7988, 0.0000, 0.2012, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:35,119 - utils - INFO - stage2_gradient_single_runtime: 0.006268501281738281
2023-09-28 23:30:35,124 - utils - INFO - 1, epoch: 1379, all client loss: [0.5220769643783569, 0.4622913897037506], all pred client disparities: [0.00998753309249878, 0.0003736764192581177], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:35,251 - utils - INFO - valid: True, epoch: 1379, loss: [0.5786888599395752, 0.46144238114356995], accuracy: [0.7292817831039429, 0.7895030975341797], mean_accuracy:0.7593924403190613,variance_accuracy:0.030110657215118408, disparity: [0.009090900421142578, 0.0044252872467041016], mean_disparity:0.00675809383392334,variance_disparity:0.0023328065872192383, pred_disparity: [0.0031188130378723145, 0.0008719116449356079]
2023-09-28 23:30:35,334 - utils - INFO - global_valid: True, epoch: 1379,  global_loss: 0.4627458453178406, global_accuracy: 0.8176446905081671,  global_disparity:0.007919088006019592, global_pred_disparity: 0.0042457133531570435,
2023-09-28 23:30:35,572 - utils - INFO - stage2_gradient_single_runtime: 0.006272077560424805
2023-09-28 23:30:35,577 - utils - INFO - 1, epoch: 1380, all client loss: [0.5220438241958618, 0.4620080590248108], all pred client disparities: [0.010076791048049927, 0.00029577314853668213], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891004085540771],  alphas:tensor([0.4806, 0.0000, 0.1396, 0.3798], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:35,814 - utils - INFO - stage2_gradient_single_runtime: 0.006285667419433594
2023-09-28 23:30:35,820 - utils - INFO - 1, epoch: 1381, all client loss: [0.5220412015914917, 0.46209147572517395], all pred client disparities: [0.010035604238510132, 0.00026966631412506104], all client disparities: [0.00688403844833374, 0.002875208854675293], all client accs: [0.7433414459228516, 0.7891315221786499],  alphas:tensor([0.9742, 0.0000, 0.0000, 0.0258], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:36,057 - utils - INFO - stage2_gradient_single_runtime: 0.00626826286315918
2023-09-28 23:30:36,062 - utils - INFO - 1, epoch: 1382, all client loss: [0.5220147967338562, 0.4618120491504669], all pred client disparities: [0.010123372077941895, 0.00017577409744262695], all client disparities: [0.00688403844833374, 0.0030213892459869385], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([0.4806, 0.0000, 0.1399, 0.3795], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:36,303 - utils - INFO - stage2_gradient_single_runtime: 0.0062503814697265625
2023-09-28 23:30:36,308 - utils - INFO - 1, epoch: 1383, all client loss: [0.5220115184783936, 0.4618959426879883], all pred client disparities: [0.010081559419631958, 0.00015085935592651367], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4806, 0.0000, 0.1399, 0.3795], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:36,551 - utils - INFO - stage2_gradient_single_runtime: 0.006280422210693359
2023-09-28 23:30:36,556 - utils - INFO - 1, epoch: 1384, all client loss: [0.5220086574554443, 0.46197935938835144], all pred client disparities: [0.01004016399383545, 0.00012513995170593262], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4806, 0.0000, 0.1399, 0.3795], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:36,793 - utils - INFO - stage2_gradient_single_runtime: 0.006245136260986328
2023-09-28 23:30:36,799 - utils - INFO - 1, epoch: 1385, all client loss: [0.5220062136650085, 0.46206238865852356], all pred client disparities: [0.009999215602874756, 9.85562801361084e-05], all client disparities: [0.00688403844833374, 0.002875208854675293], all client accs: [0.7433414459228516, 0.7891315221786499],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:37,035 - utils - INFO - stage2_gradient_single_runtime: 0.0062923431396484375
2023-09-28 23:30:37,040 - utils - INFO - 1, epoch: 1386, all client loss: [0.5219729542732239, 0.4617813229560852], all pred client disparities: [0.010088592767715454, 2.245605537609663e-05], all client disparities: [0.00688403844833374, 0.0030213892459869385], all client accs: [0.7433414459228516, 0.7894114851951599],  alphas:tensor([0.4806, 0.0000, 0.1402, 0.3792], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:37,276 - utils - INFO - stage2_gradient_single_runtime: 0.006306171417236328
2023-09-28 23:30:37,281 - utils - INFO - 1, epoch: 1387, all client loss: [0.5219699740409851, 0.46186473965644836], all pred client disparities: [0.010047048330307007, 2.9653685942321317e-06], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.4933, 0.3022, 0.2044, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:37,518 - utils - INFO - stage2_gradient_single_runtime: 0.006885528564453125
2023-09-28 23:30:37,524 - utils - INFO - 1, epoch: 1388, all client loss: [0.5219449996948242, 0.46195200085639954], all pred client disparities: [0.010008454322814941, 3.3229589462280273e-05], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([0.9753, 0.0000, 0.0000, 0.0247], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:37,778 - utils - INFO - stage2_gradient_single_runtime: 0.006261110305786133
2023-09-28 23:30:37,783 - utils - INFO - 1, epoch: 1389, all client loss: [0.5219186544418335, 0.4616730511188507], all pred client disparities: [0.01009628176689148, 6.04093074798584e-05], all client disparities: [0.00688403844833374, 0.0030213892459869385], all client accs: [0.7433414459228516, 0.7894114851951599],  alphas:tensor([0.4947, 0.3004, 0.2050, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:38,023 - utils - INFO - stage2_gradient_single_runtime: 0.00634312629699707
2023-09-28 23:30:38,028 - utils - INFO - 1, epoch: 1390, all client loss: [0.5218927264213562, 0.4617612957954407], all pred client disparities: [0.010056853294372559, 2.1576881408691406e-05], all client disparities: [0.00688403844833374, 0.0030213892459869385], all client accs: [0.7433414459228516, 0.7894114851951599],  alphas:tensor([0.4934, 0.3013, 0.2053, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:38,266 - utils - INFO - stage2_gradient_single_runtime: 0.006282329559326172
2023-09-28 23:30:38,272 - utils - INFO - 1, epoch: 1391, all client loss: [0.5218678116798401, 0.46184825897216797], all pred client disparities: [0.010018229484558105, 1.4796860341448337e-05], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7893181443214417],  alphas:tensor([0.4805, 0.0000, 0.1409, 0.3785], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:38,509 - utils - INFO - stage2_gradient_single_runtime: 0.00628209114074707
2023-09-28 23:30:38,514 - utils - INFO - 1, epoch: 1392, all client loss: [0.5218657851219177, 0.46193015575408936], all pred client disparities: [0.00997769832611084, 1.2621292626135983e-05], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7891937494277954],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:38,754 - utils - INFO - stage2_gradient_single_runtime: 0.0062634944915771484
2023-09-28 23:30:38,761 - utils - INFO - 1, epoch: 1393, all client loss: [0.5218334197998047, 0.4616486132144928], all pred client disparities: [0.010067194700241089, 9.129941463470459e-05], all client disparities: [0.00688403844833374, 0.0030213892459869385], all client accs: [0.7433414459228516, 0.7894114851951599],  alphas:tensor([0.7865, 0.0000, 0.2135, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:39,001 - utils - INFO - stage2_gradient_single_runtime: 0.006224632263183594
2023-09-28 23:30:39,007 - utils - INFO - 1, epoch: 1394, all client loss: [0.5216884016990662, 0.46192386746406555], all pred client disparities: [0.009986251592636108, 0.00034724175930023193], all client disparities: [0.00688403844833374, 0.0026559531688690186], all client accs: [0.7433414459228516, 0.7891626358032227],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:39,248 - utils - INFO - stage2_gradient_single_runtime: 0.006276369094848633
2023-09-28 23:30:39,255 - utils - INFO - 1, epoch: 1395, all client loss: [0.5216578841209412, 0.4616388976573944], all pred client disparities: [0.010075688362121582, 0.00026166439056396484], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4805, 0.0000, 0.1426, 0.3769], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:39,494 - utils - INFO - stage2_gradient_single_runtime: 0.00624847412109375
2023-09-28 23:30:39,500 - utils - INFO - 1, epoch: 1396, all client loss: [0.5216560363769531, 0.4617198407649994], all pred client disparities: [0.010035455226898193, 0.00023405253887176514], all client disparities: [0.00688403844833374, 0.0027290433645248413], all client accs: [0.7433414459228516, 0.7893492579460144],  alphas:tensor([0.9884, 0.0000, 0.0000, 0.0116], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:39,739 - utils - INFO - stage2_gradient_single_runtime: 0.0062868595123291016
2023-09-28 23:30:39,744 - utils - INFO - 1, epoch: 1397, all client loss: [0.5216283202171326, 0.4614381790161133], all pred client disparities: [0.010124415159225464, 0.00014284253120422363], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4804, 0.0000, 0.1429, 0.3767], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:39,979 - utils - INFO - stage2_gradient_single_runtime: 0.006254434585571289
2023-09-28 23:30:39,985 - utils - INFO - 1, epoch: 1398, all client loss: [0.5216259360313416, 0.4615195095539093], all pred client disparities: [0.010083407163619995, 0.00011646747589111328], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4804, 0.0000, 0.1429, 0.3767], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:40,226 - utils - INFO - stage2_gradient_single_runtime: 0.0062770843505859375
2023-09-28 23:30:40,232 - utils - INFO - 1, epoch: 1399, all client loss: [0.5216239094734192, 0.46160048246383667], all pred client disparities: [0.010042816400527954, 8.925795555114746e-05], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4804, 0.0000, 0.1429, 0.3767], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:40,316 - utils - INFO - valid: True, epoch: 1399, loss: [0.5786590576171875, 0.46109873056411743], accuracy: [0.7292817831039429, 0.7895652055740356], mean_accuracy:0.7594234943389893,variance_accuracy:0.030141711235046387, disparity: [0.009090900421142578, 0.0044252872467041016], mean_disparity:0.00675809383392334,variance_disparity:0.0023328065872192383, pred_disparity: [0.0034456253051757812, 0.0009585320949554443]
2023-09-28 23:30:40,443 - utils - INFO - global_valid: True, epoch: 1399,  global_loss: 0.4624056816101074, global_accuracy: 0.8179303673956367,  global_disparity:0.007919088006019592, global_pred_disparity: 0.004339262843132019,
2023-09-28 23:30:40,695 - utils - INFO - stage2_gradient_single_runtime: 0.006279468536376953
2023-09-28 23:30:40,699 - utils - INFO - 1, epoch: 1400, all client loss: [0.5216223001480103, 0.46168097853660583], all pred client disparities: [0.010002821683883667, 6.124377250671387e-05], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.9888, 0.0000, 0.0000, 0.0112], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:40,939 - utils - INFO - stage2_gradient_single_runtime: 0.006184101104736328
2023-09-28 23:30:40,945 - utils - INFO - 1, epoch: 1401, all client loss: [0.5215945243835449, 0.4613994061946869], all pred client disparities: [0.010091662406921387, 2.9787424864480272e-05], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4935, 0.2979, 0.2087, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:41,169 - utils - INFO - stage2_gradient_single_runtime: 0.006243705749511719
2023-09-28 23:30:41,173 - utils - INFO - 1, epoch: 1402, all client loss: [0.5215701460838318, 0.4614847004413605], all pred client disparities: [0.010053247213363647, 6.496906280517578e-06], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4804, 0.0000, 0.1433, 0.3763], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:41,411 - utils - INFO - stage2_gradient_single_runtime: 0.006307125091552734
2023-09-28 23:30:41,414 - utils - INFO - 1, epoch: 1403, all client loss: [0.5215684175491333, 0.4615650773048401], all pred client disparities: [0.010013043880462646, 2.1323567125364207e-05], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.7759, 0.0000, 0.2241, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:41,708 - utils - INFO - stage2_gradient_single_runtime: 0.006251335144042969
2023-09-28 23:30:41,710 - utils - INFO - 1, epoch: 1404, all client loss: [0.5214300751686096, 0.46183162927627563], all pred client disparities: [0.009936302900314331, 0.00040613114833831787], all client disparities: [0.00688403844833374, 0.004671156406402588], all client accs: [0.7433414459228516, 0.7892248034477234],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:41,945 - utils - INFO - stage2_gradient_single_runtime: 0.006200551986694336
2023-09-28 23:30:41,950 - utils - INFO - 1, epoch: 1405, all client loss: [0.521401584148407, 0.4615432024002075], all pred client disparities: [0.010025590658187866, 0.0003128349781036377], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894114851951599],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:42,188 - utils - INFO - stage2_gradient_single_runtime: 0.006269216537475586
2023-09-28 23:30:42,195 - utils - INFO - 1, epoch: 1406, all client loss: [0.5213727355003357, 0.46125879883766174], all pred client disparities: [0.010115474462509155, 0.0002235323190689087], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4803, 0.0000, 0.1449, 0.3748], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:42,431 - utils - INFO - stage2_gradient_single_runtime: 0.0062329769134521484
2023-09-28 23:30:42,435 - utils - INFO - 1, epoch: 1407, all client loss: [0.521371066570282, 0.4613383412361145], all pred client disparities: [0.010075300931930542, 0.0001958310604095459], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4803, 0.0000, 0.1449, 0.3748], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:42,666 - utils - INFO - stage2_gradient_single_runtime: 0.006191253662109375
2023-09-28 23:30:42,669 - utils - INFO - 1, epoch: 1408, all client loss: [0.5213696956634521, 0.4614175260066986], all pred client disparities: [0.01003563404083252, 0.00016738474369049072], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.9978, 0.0000, 0.0000, 0.0022], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:42,907 - utils - INFO - stage2_gradient_single_runtime: 0.006236553192138672
2023-09-28 23:30:42,912 - utils - INFO - 1, epoch: 1409, all client loss: [0.5213412046432495, 0.4611347019672394], all pred client disparities: [0.010125428438186646, 7.787346839904785e-05], all client disparities: [0.00688403844833374, 0.002875208854675293], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4802, 0.0000, 0.1453, 0.3745], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:43,172 - utils - INFO - stage2_gradient_single_runtime: 0.006242036819458008
2023-09-28 23:30:43,177 - utils - INFO - 1, epoch: 1410, all client loss: [0.5213393568992615, 0.4612143337726593], all pred client disparities: [0.01008501648902893, 5.0634145736694336e-05], all client disparities: [0.00688403844833374, 0.002875208854675293], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4802, 0.0000, 0.1452, 0.3745], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:43,413 - utils - INFO - stage2_gradient_single_runtime: 0.006264448165893555
2023-09-28 23:30:43,418 - utils - INFO - 1, epoch: 1411, all client loss: [0.5213378071784973, 0.4612935185432434], all pred client disparities: [0.010045111179351807, 2.2590160369873047e-05], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4802, 0.0000, 0.1452, 0.3746], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:43,655 - utils - INFO - stage2_gradient_single_runtime: 0.006455898284912109
2023-09-28 23:30:43,661 - utils - INFO - 1, epoch: 1412, all client loss: [0.5213366150856018, 0.4613723158836365], all pred client disparities: [0.010005533695220947, 6.2436042753688525e-06], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.7770, 0.0000, 0.0000, 0.2230], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:43,908 - utils - INFO - stage2_gradient_single_runtime: 0.006197452545166016
2023-09-28 23:30:43,913 - utils - INFO - 1, epoch: 1413, all client loss: [0.521256148815155, 0.46124982833862305], all pred client disparities: [0.010027796030044556, 0.00015023350715637207], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7893803715705872],  alphas:tensor([0.4802, 0.0000, 0.1458, 0.3739], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:44,154 - utils - INFO - stage2_gradient_single_runtime: 0.006232023239135742
2023-09-28 23:30:44,159 - utils - INFO - 1, epoch: 1414, all client loss: [0.5212551951408386, 0.4613279700279236], all pred client disparities: [0.009988605976104736, 0.00012084841728210449], all client disparities: [0.00688403844833374, 0.00280211865901947], all client accs: [0.7433414459228516, 0.7893803715705872],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:44,387 - utils - INFO - stage2_gradient_single_runtime: 0.006287813186645508
2023-09-28 23:30:44,390 - utils - INFO - 1, epoch: 1415, all client loss: [0.5212269425392151, 0.4610441327095032], all pred client disparities: [0.0100785493850708, 3.0204657377908006e-05], all client disparities: [0.00688403844833374, 0.002875208854675293], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4801, 0.0000, 0.1462, 0.3737], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:44,623 - utils - INFO - stage2_gradient_single_runtime: 0.006447553634643555
2023-09-28 23:30:44,630 - utils - INFO - 1, epoch: 1416, all client loss: [0.5212254524230957, 0.4611227512359619], all pred client disparities: [0.010038822889328003, 2.1011164790252224e-06], all client disparities: [0.00688403844833374, 0.002875208854675293], all client accs: [0.7433414459228516, 0.7894114851951599],  alphas:tensor([0.4802, 0.0000, 0.1461, 0.3737], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:44,863 - utils - INFO - stage2_gradient_single_runtime: 0.006165742874145508
2023-09-28 23:30:44,868 - utils - INFO - 1, epoch: 1417, all client loss: [0.521224319934845, 0.4612009823322296], all pred client disparities: [0.009999275207519531, 2.6777390303323045e-05], all client disparities: [0.00688403844833374, 0.002875208854675293], all client accs: [0.7433414459228516, 0.7894114851951599],  alphas:tensor([0.7667, 0.0000, 0.2333, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:45,109 - utils - INFO - stage2_gradient_single_runtime: 0.006199836730957031
2023-09-28 23:30:45,116 - utils - INFO - 1, epoch: 1418, all client loss: [0.5210927128791809, 0.4614580273628235], all pred client disparities: [0.009924620389938354, 0.0003884434700012207], all client disparities: [0.005072444677352905, 0.004744246602058411], all client accs: [0.7457627654075623, 0.7893492579460144],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:45,363 - utils - INFO - stage2_gradient_single_runtime: 0.0062563419342041016
2023-09-28 23:30:45,368 - utils - INFO - 1, epoch: 1419, all client loss: [0.5210661292076111, 0.46116897463798523], all pred client disparities: [0.01001429557800293, 0.00028924643993377686], all client disparities: [0.00688403844833374, 0.004890426993370056], all client accs: [0.7433414459228516, 0.7893803715705872],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:45,450 - utils - INFO - valid: True, epoch: 1419, loss: [0.5784128904342651, 0.4602731764316559], accuracy: [0.7292817831039429, 0.7893788814544678], mean_accuracy:0.7593303322792053,variance_accuracy:0.03004854917526245, disparity: [0.009090900421142578, 0.004277363419532776], mean_disparity:0.006684131920337677,variance_disparity:0.002406768500804901, pred_disparity: [0.003751128911972046, 0.0007013678550720215]
2023-09-28 23:30:45,586 - utils - INFO - global_valid: True, epoch: 1419,  global_loss: 0.461586594581604, global_accuracy: 0.8184545596540387,  global_disparity:0.007775828242301941, global_pred_disparity: 0.004093468189239502,
2023-09-28 23:30:45,823 - utils - INFO - stage2_gradient_single_runtime: 0.006228923797607422
2023-09-28 23:30:45,830 - utils - INFO - 1, epoch: 1420, all client loss: [0.5210392475128174, 0.4608839750289917], all pred client disparities: [0.010104507207870483, 0.0001942366361618042], all client disparities: [0.00688403844833374, 0.004963502287864685], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4801, 0.0000, 0.1477, 0.3722], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:46,062 - utils - INFO - stage2_gradient_single_runtime: 0.006204366683959961
2023-09-28 23:30:46,068 - utils - INFO - 1, epoch: 1421, all client loss: [0.5210379362106323, 0.46096155047416687], all pred client disparities: [0.01006510853767395, 0.00016576051712036133], all client disparities: [0.00688403844833374, 0.004963502287864685], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4801, 0.0000, 0.1477, 0.3722], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:46,305 - utils - INFO - stage2_gradient_single_runtime: 0.006219148635864258
2023-09-28 23:30:46,312 - utils - INFO - 1, epoch: 1422, all client loss: [0.5210371017456055, 0.4610387682914734], all pred client disparities: [0.01002606749534607, 0.00013649463653564453], all client disparities: [0.00688403844833374, 0.004963502287864685], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:46,550 - utils - INFO - stage2_gradient_single_runtime: 0.006269931793212891
2023-09-28 23:30:46,557 - utils - INFO - 1, epoch: 1423, all client loss: [0.5210099816322327, 0.4607553482055664], all pred client disparities: [0.010116368532180786, 4.303455352783203e-05], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4800, 0.0000, 0.1480, 0.3720], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:46,795 - utils - INFO - stage2_gradient_single_runtime: 0.006234645843505859
2023-09-28 23:30:46,802 - utils - INFO - 1, epoch: 1424, all client loss: [0.5210085511207581, 0.4608331024646759], all pred client disparities: [0.0100766122341156, 1.50352789205499e-05], all client disparities: [0.00688403844833374, 0.002948298817500472], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4800, 0.0000, 0.1480, 0.3720], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:47,044 - utils - INFO - stage2_gradient_single_runtime: 0.006279468536376953
2023-09-28 23:30:47,050 - utils - INFO - 1, epoch: 1425, all client loss: [0.5210074782371521, 0.4609104096889496], all pred client disparities: [0.010037332773208618, 1.3709068298339844e-05], all client disparities: [0.00688403844833374, 0.004963502287864685], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4906, 0.2939, 0.2155, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:47,292 - utils - INFO - stage2_gradient_single_runtime: 0.006231784820556641
2023-09-28 23:30:47,298 - utils - INFO - 1, epoch: 1426, all client loss: [0.5209861993789673, 0.46098974347114563], all pred client disparities: [0.010001331567764282, 1.665949821472168e-05], all client disparities: [0.00688403844833374, 0.004963502287864685], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:47,538 - utils - INFO - stage2_gradient_single_runtime: 0.006277561187744141
2023-09-28 23:30:47,545 - utils - INFO - 1, epoch: 1427, all client loss: [0.5209593176841736, 0.46070626378059387], all pred client disparities: [0.010091602802276611, 7.744133472442627e-05], all client disparities: [0.00688403844833374, 0.0030213892459869385], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.4919, 0.2920, 0.2161, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:47,785 - utils - INFO - stage2_gradient_single_runtime: 0.007277965545654297
2023-09-28 23:30:47,792 - utils - INFO - 1, epoch: 1428, all client loss: [0.5209370851516724, 0.46078675985336304], all pred client disparities: [0.010054737329483032, 4.42117489001248e-05], all client disparities: [0.00688403844833374, 0.005683958530426025], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4908, 0.2928, 0.2163, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:48,032 - utils - INFO - stage2_gradient_single_runtime: 0.007157802581787109
2023-09-28 23:30:48,038 - utils - INFO - 1, epoch: 1429, all client loss: [0.5209157466888428, 0.4608660638332367], all pred client disparities: [0.010018587112426758, 1.3187535842007492e-05], all client disparities: [0.00688403844833374, 0.005683958530426025], all client accs: [0.7433414459228516, 0.7894736528396606],  alphas:tensor([0.4899, 0.2936, 0.2165, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:48,282 - utils - INFO - stage2_gradient_single_runtime: 0.006273746490478516
2023-09-28 23:30:48,287 - utils - INFO - 1, epoch: 1430, all client loss: [0.5208951830863953, 0.4609442353248596], all pred client disparities: [0.009983152151107788, 1.56611276906915e-05], all client disparities: [0.005072444677352905, 0.0056108832359313965], all client accs: [0.7457627654075623, 0.7895047664642334],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:48,472 - utils - INFO - stage2_gradient_single_runtime: 0.006859540939331055
2023-09-28 23:30:48,478 - utils - INFO - 1, epoch: 1431, all client loss: [0.5208688974380493, 0.46065980195999146], all pred client disparities: [0.010073423385620117, 8.088350296020508e-05], all client disparities: [0.00688403844833374, 0.005757048726081848], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.7605, 0.0000, 0.2395, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:48,725 - utils - INFO - stage2_gradient_single_runtime: 0.006213665008544922
2023-09-28 23:30:48,731 - utils - INFO - 1, epoch: 1432, all client loss: [0.5207422971725464, 0.4609089493751526], all pred client disparities: [0.009998470544815063, 0.00032401084899902344], all client disparities: [0.005072444677352905, 0.005537793040275574], all client accs: [0.7457627654075623, 0.7894736528396606],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:48,969 - utils - INFO - stage2_gradient_single_runtime: 0.006283998489379883
2023-09-28 23:30:48,976 - utils - INFO - 1, epoch: 1433, all client loss: [0.5207173228263855, 0.4606219232082367], all pred client disparities: [0.010088860988616943, 0.00022195279598236084], all client disparities: [0.00688403844833374, 0.005537793040275574], all client accs: [0.7433414459228516, 0.7895047664642334],  alphas:tensor([0.4800, 0.0000, 0.1504, 0.3697], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:49,213 - utils - INFO - stage2_gradient_single_runtime: 0.00627589225769043
2023-09-28 23:30:49,219 - utils - INFO - 1, epoch: 1434, all client loss: [0.5207167267799377, 0.4606974720954895], all pred client disparities: [0.010050475597381592, 0.00019225478172302246], all client disparities: [0.00688403844833374, 0.005537793040275574], all client accs: [0.7433414459228516, 0.7895047664642334],  alphas:tensor([0.4800, 0.0000, 0.1503, 0.3697], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:49,455 - utils - INFO - stage2_gradient_single_runtime: 0.006258964538574219
2023-09-28 23:30:49,461 - utils - INFO - 1, epoch: 1435, all client loss: [0.5207164883613586, 0.4607726037502289], all pred client disparities: [0.010012388229370117, 0.0001618117094039917], all client disparities: [0.005072444677352905, 0.005537793040275574], all client accs: [0.7457627654075623, 0.7895047664642334],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:49,699 - utils - INFO - stage2_gradient_single_runtime: 0.006362438201904297
2023-09-28 23:30:49,706 - utils - INFO - 1, epoch: 1436, all client loss: [0.5206913948059082, 0.46048736572265625], all pred client disparities: [0.01010286808013916, 6.158649921417236e-05], all client disparities: [0.00688403844833374, 0.005976319313049316], all client accs: [0.7433414459228516, 0.7893803715705872],  alphas:tensor([0.4799, 0.0000, 0.1507, 0.3695], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:49,943 - utils - INFO - stage2_gradient_single_runtime: 0.006257534027099609
2023-09-28 23:30:49,950 - utils - INFO - 1, epoch: 1437, all client loss: [0.5206905603408813, 0.460563063621521], all pred client disparities: [0.010064184665679932, 3.24249267578125e-05], all client disparities: [0.00688403844833374, 0.005903229117393494], all client accs: [0.7433414459228516, 0.7894114851951599],  alphas:tensor([0.4799, 0.0000, 0.1506, 0.3695], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:50,187 - utils - INFO - stage2_gradient_single_runtime: 0.006256103515625
2023-09-28 23:30:50,195 - utils - INFO - 1, epoch: 1438, all client loss: [0.5206900238990784, 0.4606383442878723], all pred client disparities: [0.010025829076766968, 2.5331974029541016e-06], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894114851951599],  alphas:tensor([0.4799, 0.0000, 0.1505, 0.3695], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:50,435 - utils - INFO - stage2_gradient_single_runtime: 0.006269216537475586
2023-09-28 23:30:50,442 - utils - INFO - 1, epoch: 1439, all client loss: [0.520689845085144, 0.46071329712867737], all pred client disparities: [0.009987801313400269, 2.808869248838164e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894114851951599],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:50,574 - utils - INFO - valid: True, epoch: 1439, loss: [0.5783277750015259, 0.45979639887809753], accuracy: [0.7292817831039429, 0.7892546653747559], mean_accuracy:0.7592682242393494,variance_accuracy:0.029986441135406494, disparity: [0.009090900421142578, 0.004573225975036621], mean_disparity:0.0068320631980896,variance_disparity:0.0022588372230529785, pred_disparity: [0.004014939069747925, 0.0008792281150817871]
2023-09-28 23:30:50,654 - utils - INFO - global_valid: True, epoch: 1439,  global_loss: 0.46111416816711426, global_accuracy: 0.8187400379006717,  global_disparity:0.008062362670898438, global_pred_disparity: 0.004274576902389526,
2023-09-28 23:30:50,889 - utils - INFO - stage2_gradient_single_runtime: 0.006245136260986328
2023-09-28 23:30:50,893 - utils - INFO - 1, epoch: 1440, all client loss: [0.520664632320404, 0.46042871475219727], all pred client disparities: [0.010078340768814087, 0.0001277327537536621], all client disparities: [0.00688403844833374, 0.005976319313049316], all client accs: [0.7433414459228516, 0.7894425988197327],  alphas:tensor([0.7557, 0.0000, 0.2443, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:51,130 - utils - INFO - stage2_gradient_single_runtime: 0.006254673004150391
2023-09-28 23:30:51,134 - utils - INFO - 1, epoch: 1441, all client loss: [0.5205414295196533, 0.4606728255748749], all pred client disparities: [0.010004550218582153, 0.00027020275592803955], all client disparities: [0.005072444677352905, 0.005537793040275574], all client accs: [0.7457627654075623, 0.7894736528396606],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:51,371 - utils - INFO - stage2_gradient_single_runtime: 0.00626373291015625
2023-09-28 23:30:51,375 - utils - INFO - 1, epoch: 1442, all client loss: [0.5205175280570984, 0.460385799407959], all pred client disparities: [0.010095179080963135, 0.00016532838344573975], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4798, 0.0000, 0.1521, 0.3681], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:51,610 - utils - INFO - stage2_gradient_single_runtime: 0.006250143051147461
2023-09-28 23:30:51,616 - utils - INFO - 1, epoch: 1443, all client loss: [0.520517110824585, 0.4604603052139282], all pred client disparities: [0.010057061910629272, 0.00013546645641326904], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894114851951599],  alphas:tensor([0.4799, 0.0000, 0.1520, 0.3681], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:51,860 - utils - INFO - stage2_gradient_single_runtime: 0.006294727325439453
2023-09-28 23:30:51,865 - utils - INFO - 1, epoch: 1444, all client loss: [0.5205169916152954, 0.4605344831943512], all pred client disparities: [0.010019361972808838, 0.00010485947132110596], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7893803715705872],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:52,098 - utils - INFO - stage2_gradient_single_runtime: 0.006277561187744141
2023-09-28 23:30:52,103 - utils - INFO - 1, epoch: 1445, all client loss: [0.5204927921295166, 0.4602493345737457], all pred client disparities: [0.010110169649124146, 1.9371509552001953e-06], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894736528396606],  alphas:tensor([0.4797, 0.0000, 0.1524, 0.3679], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:52,336 - utils - INFO - stage2_gradient_single_runtime: 0.006293773651123047
2023-09-28 23:30:52,341 - utils - INFO - 1, epoch: 1446, all client loss: [0.5204921364784241, 0.4603239893913269], all pred client disparities: [0.010071724653244019, 2.737343675107695e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4895, 0.2889, 0.2216, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:52,573 - utils - INFO - stage2_gradient_single_runtime: 0.0062580108642578125
2023-09-28 23:30:52,578 - utils - INFO - 1, epoch: 1447, all client loss: [0.5204720497131348, 0.4604000747203827], all pred client disparities: [0.010036736726760864, 1.4156887573335553e-06], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4798, 0.0000, 0.1524, 0.3678], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:52,811 - utils - INFO - stage2_gradient_single_runtime: 0.006241559982299805
2023-09-28 23:30:52,816 - utils - INFO - 1, epoch: 1448, all client loss: [0.5204720497131348, 0.4604739248752594], all pred client disparities: [0.009999096393585205, 2.9414892196655273e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:53,055 - utils - INFO - stage2_gradient_single_runtime: 0.007180452346801758
2023-09-28 23:30:53,062 - utils - INFO - 1, epoch: 1449, all client loss: [0.5204479694366455, 0.46018901467323303], all pred client disparities: [0.010090023279190063, 0.0001324862241744995], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.7506, 0.0000, 0.2494, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:53,298 - utils - INFO - stage2_gradient_single_runtime: 0.006358146667480469
2023-09-28 23:30:53,305 - utils - INFO - 1, epoch: 1450, all client loss: [0.5203284025192261, 0.4604278802871704], all pred client disparities: [0.010017335414886475, 0.00025819242000579834], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:53,547 - utils - INFO - stage2_gradient_single_runtime: 0.00634455680847168
2023-09-28 23:30:53,553 - utils - INFO - 1, epoch: 1451, all client loss: [0.5203055143356323, 0.4601407051086426], all pred client disparities: [0.010108321905136108, 0.00015012919902801514], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4797, 0.0000, 0.1540, 0.3663], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:53,878 - utils - INFO - stage2_gradient_single_runtime: 0.0062754154205322266
2023-09-28 23:30:53,883 - utils - INFO - 1, epoch: 1452, all client loss: [0.5203051567077637, 0.4602142870426178], all pred client disparities: [0.010070562362670898, 0.00012020766735076904], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4798, 0.0000, 0.1539, 0.3663], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:54,112 - utils - INFO - stage2_gradient_single_runtime: 0.006224155426025391
2023-09-28 23:30:54,116 - utils - INFO - 1, epoch: 1453, all client loss: [0.5203051567077637, 0.4602874517440796], all pred client disparities: [0.010033130645751953, 8.955597877502441e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4798, 0.0000, 0.1538, 0.3664], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:54,349 - utils - INFO - stage2_gradient_single_runtime: 0.006205081939697266
2023-09-28 23:30:54,353 - utils - INFO - 1, epoch: 1454, all client loss: [0.5203053951263428, 0.46036025881767273], all pred client disparities: [0.009996145963668823, 5.82188404223416e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:54,583 - utils - INFO - stage2_gradient_single_runtime: 0.0062732696533203125
2023-09-28 23:30:54,587 - utils - INFO - 1, epoch: 1455, all client loss: [0.5202823281288147, 0.4600740075111389], all pred client disparities: [0.010086983442306519, 4.889071351499297e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4891, 0.2868, 0.2241, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:54,862 - utils - INFO - stage2_gradient_single_runtime: 0.006235361099243164
2023-09-28 23:30:54,867 - utils - INFO - 1, epoch: 1456, all client loss: [0.5202626585960388, 0.46014872193336487], all pred client disparities: [0.01005244255065918, 2.0578509065671824e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4882, 0.2876, 0.2242, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:55,110 - utils - INFO - stage2_gradient_single_runtime: 0.010086774826049805
2023-09-28 23:30:55,115 - utils - INFO - 1, epoch: 1457, all client loss: [0.520243763923645, 0.4602224826812744], all pred client disparities: [0.010018467903137207, 5.751848220825195e-06], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4798, 0.0000, 0.1543, 0.3659], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:55,379 - utils - INFO - stage2_gradient_single_runtime: 0.006226062774658203
2023-09-28 23:30:55,385 - utils - INFO - 1, epoch: 1458, all client loss: [0.5202441215515137, 0.4602949321269989], all pred client disparities: [0.00998157262802124, 2.5838613510131836e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:55,657 - utils - INFO - stage2_gradient_single_runtime: 0.006249427795410156
2023-09-28 23:30:55,664 - utils - INFO - 1, epoch: 1459, all client loss: [0.5202213525772095, 0.4600086212158203], all pred client disparities: [0.010072529315948486, 0.00013370811939239502], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.7444, 0.0000, 0.2556, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:55,744 - utils - INFO - valid: True, epoch: 1459, loss: [0.5783056020736694, 0.45960116386413574], accuracy: [0.7348066568374634, 0.7892546653747559], mean_accuracy:0.7620306611061096,variance_accuracy:0.02722400426864624, disparity: [0.004545450210571289, 0.004277363419532776], mean_disparity:0.0044114068150520325,variance_disparity:0.0001340433955192566, pred_disparity: [0.0041138529777526855, 0.0004379153251647949]
2023-09-28 23:30:55,916 - utils - INFO - global_valid: True, epoch: 1459,  global_loss: 0.4609208405017853, global_accuracy: 0.8191589086965168,  global_disparity:0.007632553577423096, global_pred_disparity: 0.003836393356323242,
2023-09-28 23:30:56,162 - utils - INFO - stage2_gradient_single_runtime: 0.009546995162963867
2023-09-28 23:30:56,166 - utils - INFO - 1, epoch: 1460, all client loss: [0.5201058387756348, 0.4602416753768921], all pred client disparities: [0.010002195835113525, 0.00024856626987457275], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7893803715705872],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:56,427 - utils - INFO - stage2_gradient_single_runtime: 0.0063648223876953125
2023-09-28 23:30:56,430 - utils - INFO - 1, epoch: 1461, all client loss: [0.5200841426849365, 0.45995330810546875], all pred client disparities: [0.010093361139297485, 0.00013609230518341064], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4797, 0.0000, 0.1559, 0.3645], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:56,709 - utils - INFO - stage2_gradient_single_runtime: 0.006242513656616211
2023-09-28 23:30:56,715 - utils - INFO - 1, epoch: 1462, all client loss: [0.5200840830802917, 0.4600255787372589], all pred client disparities: [0.010056257247924805, 0.00010563433170318604], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4797, 0.0000, 0.1558, 0.3645], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:56,989 - utils - INFO - stage2_gradient_single_runtime: 0.008599519729614258
2023-09-28 23:30:56,994 - utils - INFO - 1, epoch: 1463, all client loss: [0.5200842618942261, 0.4600975215435028], all pred client disparities: [0.010019510984420776, 7.449090480804443e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:57,269 - utils - INFO - stage2_gradient_single_runtime: 0.006291866302490234
2023-09-28 23:30:57,274 - utils - INFO - 1, epoch: 1464, all client loss: [0.5200623869895935, 0.45981118083000183], all pred client disparities: [0.010110795497894287, 3.571808701963164e-05], all client disparities: [0.005072444677352905, 0.005976319313049316], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4887, 0.2845, 0.2268, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:57,510 - utils - INFO - stage2_gradient_single_runtime: 0.006302595138549805
2023-09-28 23:30:57,515 - utils - INFO - 1, epoch: 1465, all client loss: [0.5200430750846863, 0.45988476276397705], all pred client disparities: [0.010076552629470825, 7.763519533909857e-06], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4878, 0.2853, 0.2269, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:57,794 - utils - INFO - stage2_gradient_single_runtime: 0.006257534027099609
2023-09-28 23:30:57,799 - utils - INFO - 1, epoch: 1466, all client loss: [0.5200244784355164, 0.4599573016166687], all pred client disparities: [0.01004287600517273, 1.825392791943159e-05], all client disparities: [0.005072444677352905, 0.005903229117393494], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.4797, 0.0000, 0.1563, 0.3640], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:58,078 - utils - INFO - stage2_gradient_single_runtime: 0.007457733154296875
2023-09-28 23:30:58,083 - utils - INFO - 1, epoch: 1467, all client loss: [0.5200247764587402, 0.46002882719039917], all pred client disparities: [0.010006338357925415, 1.306832655245671e-05], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.7526, 0.0000, 0.0000, 0.2474], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:58,323 - utils - INFO - stage2_gradient_single_runtime: 0.007470130920410156
2023-09-28 23:30:58,328 - utils - INFO - 1, epoch: 1468, all client loss: [0.5199503302574158, 0.4599210321903229], all pred client disparities: [0.010021299123764038, 0.00015057623386383057], all client disparities: [0.005072444677352905, 0.005757048726081848], all client accs: [0.7457627654075623, 0.7893492579460144],  alphas:tensor([0.4797, 0.0000, 0.1569, 0.3633], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:58,568 - utils - INFO - stage2_gradient_single_runtime: 0.006451129913330078
2023-09-28 23:30:58,574 - utils - INFO - 1, epoch: 1469, all client loss: [0.519950807094574, 0.45999208092689514], all pred client disparities: [0.009985148906707764, 0.00011894106864929199], all client disparities: [0.005072444677352905, 0.005757048726081848], all client accs: [0.7457627654075623, 0.7893492579460144],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:58,810 - utils - INFO - stage2_gradient_single_runtime: 0.006257295608520508
2023-09-28 23:30:58,813 - utils - INFO - 1, epoch: 1470, all client loss: [0.5199296474456787, 0.4597046673297882], all pred client disparities: [0.010076552629470825, 5.6177573242166545e-06], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7894114851951599],  alphas:tensor([0.4795, 0.0000, 0.1573, 0.3632], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:59,051 - utils - INFO - stage2_gradient_single_runtime: 0.006351947784423828
2023-09-28 23:30:59,057 - utils - INFO - 1, epoch: 1471, all client loss: [0.5199295282363892, 0.4597763121128082], all pred client disparities: [0.01003962755203247, 2.4691227736184373e-05], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7893803715705872],  alphas:tensor([0.4871, 0.2847, 0.2282, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:59,294 - utils - INFO - stage2_gradient_single_runtime: 0.006262302398681641
2023-09-28 23:30:59,300 - utils - INFO - 1, epoch: 1472, all client loss: [0.519911527633667, 0.4598475396633148], all pred client disparities: [0.010006576776504517, 1.1920928955078125e-07], all client disparities: [0.005072444677352905, 0.005757048726081848], all client accs: [0.7457627654075623, 0.7893492579460144],  alphas:tensor([0.4863, 0.2854, 0.2283, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:59,537 - utils - INFO - stage2_gradient_single_runtime: 0.006290435791015625
2023-09-28 23:30:59,543 - utils - INFO - 1, epoch: 1473, all client loss: [0.5198941230773926, 0.45991793274879456], all pred client disparities: [0.009974122047424316, 2.256035804748535e-05], all client disparities: [0.005072444677352905, 0.005757048726081848], all client accs: [0.7457627654075623, 0.7893181443214417],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:30:59,775 - utils - INFO - stage2_gradient_single_runtime: 0.006271839141845703
2023-09-28 23:30:59,781 - utils - INFO - 1, epoch: 1474, all client loss: [0.5198730826377869, 0.45963072776794434], all pred client disparities: [0.010065525770187378, 9.113550186157227e-05], all client disparities: [0.005072444677352905, 0.006049409508705139], all client accs: [0.7457627654075623, 0.7894425988197327],  alphas:tensor([0.7361, 0.0000, 0.2639, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:00,016 - utils - INFO - stage2_gradient_single_runtime: 0.006397247314453125
2023-09-28 23:31:00,022 - utils - INFO - 1, epoch: 1475, all client loss: [0.5197630524635315, 0.4598556160926819], all pred client disparities: [0.009997695684432983, 0.0002789497375488281], all client disparities: [0.005072444677352905, 0.005757048726081848], all client accs: [0.7457627654075623, 0.7891937494277954],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:00,268 - utils - INFO - stage2_gradient_single_runtime: 0.006400585174560547
2023-09-28 23:31:00,273 - utils - INFO - 1, epoch: 1476, all client loss: [0.519743025302887, 0.45956671237945557], all pred client disparities: [0.010089248418807983, 0.0001611560583114624], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7893492579460144],  alphas:tensor([0.4796, 0.0000, 0.1590, 0.3615], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:00,513 - utils - INFO - stage2_gradient_single_runtime: 0.006310462951660156
2023-09-28 23:31:00,518 - utils - INFO - 1, epoch: 1477, all client loss: [0.5197430849075317, 0.45963743329048157], all pred client disparities: [0.010052770376205444, 0.00013080239295959473], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7892870306968689],  alphas:tensor([0.4796, 0.0000, 0.1589, 0.3615], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:00,761 - utils - INFO - stage2_gradient_single_runtime: 0.006463766098022461
2023-09-28 23:31:00,765 - utils - INFO - 1, epoch: 1478, all client loss: [0.5197433829307556, 0.4597078263759613], all pred client disparities: [0.010016769170761108, 9.973347187042236e-05], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7892248034477234],  alphas:tensor([0.4796, 0.0000, 0.1588, 0.3616], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:01,005 - utils - INFO - stage2_gradient_single_runtime: 0.0063343048095703125
2023-09-28 23:31:01,010 - utils - INFO - 1, epoch: 1479, all client loss: [0.5197439193725586, 0.4597778916358948], all pred client disparities: [0.00998106598854065, 6.80387020111084e-05], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7892248034477234],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:01,090 - utils - INFO - valid: True, epoch: 1479, loss: [0.5780895352363586, 0.45881977677345276], accuracy: [0.7348066568374634, 0.7895030975341797], mean_accuracy:0.7621548771858215,variance_accuracy:0.027348220348358154, disparity: [0.004545450210571289, 0.0044252872467041016], mean_disparity:0.004485368728637695,variance_disparity:6.008148193359375e-05, pred_disparity: [0.004372119903564453, 0.0005986243486404419]
2023-09-28 23:31:01,230 - utils - INFO - global_valid: True, epoch: 1479,  global_loss: 0.4601457715034485, global_accuracy: 0.8196554585142795,  global_disparity:0.007775828242301941, global_pred_disparity: 0.00400136411190033,
2023-09-28 23:31:01,470 - utils - INFO - stage2_gradient_single_runtime: 0.006445646286010742
2023-09-28 23:31:01,476 - utils - INFO - 1, epoch: 1480, all client loss: [0.5197237133979797, 0.4594901204109192], all pred client disparities: [0.010072648525238037, 4.842877388000488e-05], all client disparities: [0.005072444677352905, 0.006049409508705139], all client accs: [0.7457627654075623, 0.7893492579460144],  alphas:tensor([0.4870, 0.2822, 0.2308, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:01,715 - utils - INFO - stage2_gradient_single_runtime: 0.006227016448974609
2023-09-28 23:31:01,722 - utils - INFO - 1, epoch: 1481, all client loss: [0.5197057127952576, 0.4595606327056885], all pred client disparities: [0.010039716958999634, 2.340972969250288e-05], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7893181443214417],  alphas:tensor([0.4862, 0.2830, 0.2308, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:01,907 - utils - INFO - stage2_gradient_single_runtime: 0.007045745849609375
2023-09-28 23:31:01,912 - utils - INFO - 1, epoch: 1482, all client loss: [0.5196883082389832, 0.45963019132614136], all pred client disparities: [0.010007381439208984, 2.240135756892414e-07], all client disparities: [0.005072444677352905, 0.005830138921737671], all client accs: [0.7457627654075623, 0.7892559170722961],  alphas:tensor([0.4856, 0.2833, 0.2311, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:02,148 - utils - INFO - stage2_gradient_single_runtime: 0.00691676139831543
2023-09-28 23:31:02,153 - utils - INFO - 1, epoch: 1483, all client loss: [0.5196713209152222, 0.4596991539001465], all pred client disparities: [0.009975522756576538, 2.2023916244506836e-05], all client disparities: [0.005072444677352905, 0.005757048726081848], all client accs: [0.7457627654075623, 0.7892870306968689],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:02,404 - utils - INFO - stage2_gradient_single_runtime: 0.006434917449951172
2023-09-28 23:31:02,408 - utils - INFO - 1, epoch: 1484, all client loss: [0.5196513533592224, 0.4594113230705261], all pred client disparities: [0.010067284107208252, 9.530782699584961e-05], all client disparities: [0.005072444677352905, 0.005548223853111267], all client accs: [0.7457627654075623, 0.7892559170722961],  alphas:tensor([0.7310, 0.0000, 0.2690, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:02,645 - utils - INFO - stage2_gradient_single_runtime: 0.006203174591064453
2023-09-28 23:31:02,648 - utils - INFO - 1, epoch: 1485, all client loss: [0.5195446610450745, 0.4596312344074249], all pred client disparities: [0.010001003742218018, 0.0002670884132385254], all client disparities: [0.005072444677352905, 0.000264972448348999], all client accs: [0.7457627654075623, 0.7914333939552307],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:02,881 - utils - INFO - stage2_gradient_single_runtime: 0.00626373291015625
2023-09-28 23:31:02,884 - utils - INFO - 1, epoch: 1486, all client loss: [0.5195256471633911, 0.459341824054718], all pred client disparities: [0.010092765092849731, 0.00014594197273254395], all client disparities: [0.005072444677352905, 0.00041115283966064453], all client accs: [0.7457627654075623, 0.7915889024734497],  alphas:tensor([0.4795, 0.0000, 0.1610, 0.3596], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:03,113 - utils - INFO - stage2_gradient_single_runtime: 0.006220817565917969
2023-09-28 23:31:03,118 - utils - INFO - 1, epoch: 1487, all client loss: [0.5195257067680359, 0.4594115912914276], all pred client disparities: [0.010056823492050171, 0.00011563301086425781], all client disparities: [0.005072444677352905, 0.0003380626440048218], all client accs: [0.7457627654075623, 0.791557788848877],  alphas:tensor([0.4795, 0.0000, 0.1609, 0.3596], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:03,352 - utils - INFO - stage2_gradient_single_runtime: 0.006273746490478516
2023-09-28 23:31:03,357 - utils - INFO - 1, epoch: 1488, all client loss: [0.5195260643959045, 0.45948106050491333], all pred client disparities: [0.010021239519119263, 8.469820022583008e-05], all client disparities: [0.005072444677352905, 0.000264972448348999], all client accs: [0.7457627654075623, 0.7914955615997314],  alphas:tensor([0.4796, 0.0000, 0.1608, 0.3596], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:03,593 - utils - INFO - stage2_gradient_single_runtime: 0.006275653839111328
2023-09-28 23:31:03,598 - utils - INFO - 1, epoch: 1489, all client loss: [0.5195266604423523, 0.4595501720905304], all pred client disparities: [0.00998598337173462, 5.31226432940457e-05], all client disparities: [0.005072444677352905, 0.000264972448348999], all client accs: [0.7457627654075623, 0.7914955615997314],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:03,831 - utils - INFO - stage2_gradient_single_runtime: 0.006250143051147461
2023-09-28 23:31:03,836 - utils - INFO - 1, epoch: 1490, all client loss: [0.5195073485374451, 0.45926204323768616], all pred client disparities: [0.010077744722366333, 6.653368473052979e-05], all client disparities: [0.005072444677352905, 0.005475133657455444], all client accs: [0.7457627654075623, 0.7892559170722961],  alphas:tensor([0.4863, 0.2804, 0.2334, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:04,071 - utils - INFO - stage2_gradient_single_runtime: 0.006229400634765625
2023-09-28 23:31:04,076 - utils - INFO - 1, epoch: 1491, all client loss: [0.5194898843765259, 0.45933109521865845], all pred client disparities: [0.010045498609542847, 4.246830940246582e-05], all client disparities: [0.005072444677352905, 0.005328953266143799], all client accs: [0.7457627654075623, 0.7892248034477234],  alphas:tensor([0.4855, 0.2811, 0.2334, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:04,310 - utils - INFO - stage2_gradient_single_runtime: 0.0062618255615234375
2023-09-28 23:31:04,315 - utils - INFO - 1, epoch: 1492, all client loss: [0.5194729566574097, 0.4593992531299591], all pred client disparities: [0.010013729333877563, 2.0191078874631785e-05], all client disparities: [0.005072444677352905, 1.6957521438598633e-05], all client accs: [0.7457627654075623, 0.7914644479751587],  alphas:tensor([0.7245, 0.0000, 0.2755, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:04,547 - utils - INFO - stage2_gradient_single_runtime: 0.006208658218383789
2023-09-28 23:31:04,552 - utils - INFO - 1, epoch: 1493, all client loss: [0.5193699598312378, 0.4596141576766968], all pred client disparities: [0.009950816631317139, 0.0003342926502227783], all client disparities: [0.005072444677352905, 0.00019188225269317627], all client accs: [0.7457627654075623, 0.7914333939552307],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:04,786 - utils - INFO - stage2_gradient_single_runtime: 0.006217241287231445
2023-09-28 23:31:04,791 - utils - INFO - 1, epoch: 1494, all client loss: [0.5193519592285156, 0.45932164788246155], all pred client disparities: [0.010042428970336914, 0.0002074986696243286], all client disparities: [0.005072444677352905, 0.00019188225269317627], all client accs: [0.7457627654075623, 0.7915266752243042],  alphas:tensor([0.4796, 0.0000, 0.1624, 0.3580], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:05,022 - utils - INFO - stage2_gradient_single_runtime: 0.006214618682861328
2023-09-28 23:31:05,027 - utils - INFO - 1, epoch: 1495, all client loss: [0.5193524360656738, 0.45939016342163086], all pred client disparities: [0.010007500648498535, 0.00017625093460083008], all client disparities: [0.005072444677352905, 0.00019188225269317627], all client accs: [0.7457627654075623, 0.7915266752243042],  alphas:tensor([1., 0., 0., 0.], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:05,259 - utils - INFO - stage2_gradient_single_runtime: 0.006253719329833984
2023-09-28 23:31:05,264 - utils - INFO - 1, epoch: 1496, all client loss: [0.519334077835083, 0.4591011703014374], all pred client disparities: [0.010099619626998901, 5.3256750106811523e-05], all client disparities: [0.005072444677352905, 9.004771709442139e-05], all client accs: [0.7457627654075623, 0.7914644479751587],  alphas:tensor([0.4794, 0.0000, 0.1627, 0.3579], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:05,499 - utils - INFO - stage2_gradient_single_runtime: 0.006241798400878906
2023-09-28 23:31:05,504 - utils - INFO - 1, epoch: 1497, all client loss: [0.519334077835083, 0.45917028188705444], all pred client disparities: [0.010063916444778442, 2.3320322725339793e-05], all client disparities: [0.005072444677352905, 9.004771709442139e-05], all client accs: [0.7457627654075623, 0.7914644479751587],  alphas:tensor([0.4794, 0.0000, 0.1627, 0.3579], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:05,738 - utils - INFO - stage2_gradient_single_runtime: 0.006243705749511719
2023-09-28 23:31:05,743 - utils - INFO - 1, epoch: 1498, all client loss: [0.5193343758583069, 0.45923906564712524], all pred client disparities: [0.01002851128578186, 7.256880962813739e-06], all client disparities: [0.005072444677352905, 9.004771709442139e-05], all client accs: [0.7457627654075623, 0.7914644479751587],  alphas:tensor([0.4845, 0.2803, 0.2351, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:05,977 - utils - INFO - stage2_gradient_single_runtime: 0.0062024593353271484
2023-09-28 23:31:05,982 - utils - INFO - 1, epoch: 1499, all client loss: [0.5193181037902832, 0.45930564403533936], all pred client disparities: [0.00999760627746582, 1.3217338164395187e-05], all client disparities: [0.005072444677352905, 9.004771709442139e-05], all client accs: [0.7457627654075623, 0.7914644479751587],  alphas:tensor([0.7203, 0.0000, 0.2797, 0.0000], device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:06,112 - utils - INFO - valid: True, epoch: 1499, loss: [0.5781517624855042, 0.45884525775909424], accuracy: [0.7348066568374634, 0.791242241859436], mean_accuracy:0.7630244493484497,variance_accuracy:0.028217792510986328, disparity: [0.004545450210571289, 0.0013439357280731201], mean_disparity:0.0029446929693222046,variance_disparity:0.0016007572412490845, pred_disparity: [0.004366129636764526, 0.00016057491302490234]
2023-09-28 23:31:06,189 - utils - INFO - global_valid: True, epoch: 1499,  global_loss: 0.460171639919281, global_accuracy: 0.8200036027174904,  global_disparity:0.0021884292364120483, global_pred_disparity: 0.00356079638004303,
2023-09-28 23:31:06,190 - utils - INFO - stage2_runtime: 191.5221266746521
2023-09-28 23:31:06,424 - utils - INFO - stage3_gradient_single_runtime: 0.0068933963775634766
2023-09-28 23:31:06,429 - utils - INFO - 1, epoch: 1500, all client loss: [0.5192177295684814, 0.45951685309410095], all pred client disparities: [0.009936541318893433, 0.00036175549030303955], all client disparities: [0.0032608509063720703, 0.00019188225269317627], all client accs: [0.7457627654075623, 0.7914333939552307],alphas:tensor([0.8618, 0.1382, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:06,662 - utils - INFO - stage3_gradient_single_runtime: 0.006297588348388672
2023-09-28 23:31:06,667 - utils - INFO - 1, epoch: 1501, all client loss: [0.5191059708595276, 0.4595174789428711], all pred client disparities: [0.00994214415550232, 0.0006160736083984375], all client disparities: [0.0032608509063720703, 0.0006930828094482422], all client accs: [0.7457627654075623, 0.7912467122077942],alphas:tensor([0.8637, 0.1363, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:06,899 - utils - INFO - stage3_gradient_single_runtime: 0.0062868595123291016
2023-09-28 23:31:06,904 - utils - INFO - 1, epoch: 1502, all client loss: [0.5189957618713379, 0.45951810479164124], all pred client disparities: [0.00994846224784851, 0.0008672326803207397], all client disparities: [0.0032608509063720703, 0.0006930828094482422], all client accs: [0.7457627654075623, 0.7912155985832214],alphas:tensor([0.8654, 0.1346, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:07,137 - utils - INFO - stage3_gradient_single_runtime: 0.0062825679779052734
2023-09-28 23:31:07,142 - utils - INFO - 1, epoch: 1503, all client loss: [0.5188872218132019, 0.4595187306404114], all pred client disparities: [0.009955346584320068, 0.0011153072118759155], all client disparities: [0.0032608509063720703, 0.0007686913013458252], all client accs: [0.7457627654075623, 0.7913711667060852],alphas:tensor([0.8672, 0.1328, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:07,375 - utils - INFO - stage3_gradient_single_runtime: 0.006262302398681641
2023-09-28 23:31:07,380 - utils - INFO - 1, epoch: 1504, all client loss: [0.5187802314758301, 0.4595193564891815], all pred client disparities: [0.009962856769561768, 0.001360192894935608], all client disparities: [0.0032608509063720703, 0.000841781497001648], all client accs: [0.7457627654075623, 0.7913089394569397],alphas:tensor([0.8690, 0.1310, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:07,613 - utils - INFO - stage3_gradient_single_runtime: 0.006276845932006836
2023-09-28 23:31:07,617 - utils - INFO - 1, epoch: 1505, all client loss: [0.5186747312545776, 0.4595199525356293], all pred client disparities: [0.009970873594284058, 0.0016019642353057861], all client disparities: [0.0032608509063720703, 0.000841781497001648], all client accs: [0.7457627654075623, 0.7913089394569397],alphas:tensor([0.8707, 0.1293, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:07,848 - utils - INFO - stage3_gradient_single_runtime: 0.006317615509033203
2023-09-28 23:31:07,853 - utils - INFO - 1, epoch: 1506, all client loss: [0.5185706615447998, 0.45952051877975464], all pred client disparities: [0.00997951626777649, 0.0018405914306640625], all client disparities: [0.0032608509063720703, 0.0010610520839691162], all client accs: [0.7457627654075623, 0.7913711667060852],alphas:tensor([0.8724, 0.1276, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:08,087 - utils - INFO - stage3_gradient_single_runtime: 0.0062999725341796875
2023-09-28 23:31:08,092 - utils - INFO - 1, epoch: 1507, all client loss: [0.5184679627418518, 0.4595211446285248], all pred client disparities: [0.009988635778427124, 0.002076059579849243], all client disparities: [0.0032608509063720703, 0.008108839392662048], all client accs: [0.7457627654075623, 0.792397677898407],alphas:tensor([0.8740, 0.1260, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:08,325 - utils - INFO - stage3_gradient_single_runtime: 0.00629425048828125
2023-09-28 23:31:08,330 - utils - INFO - 1, epoch: 1508, all client loss: [0.5183667540550232, 0.45952171087265015], all pred client disparities: [0.009998321533203125, 0.0023084133863449097], all client disparities: [0.0032608509063720703, 0.008474290370941162], all client accs: [0.7457627654075623, 0.791837751865387],alphas:tensor([0.8757, 0.1243, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:08,566 - utils - INFO - stage3_gradient_single_runtime: 0.007413387298583984
2023-09-28 23:31:08,574 - utils - INFO - 1, epoch: 1509, all client loss: [0.5182667970657349, 0.4595223069190979], all pred client disparities: [0.010008394718170166, 0.0025376081466674805], all client disparities: [0.0032608509063720703, 0.007973089814186096], all client accs: [0.7457627654075623, 0.791713297367096],alphas:tensor([0.5918, 0.4082, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:08,812 - utils - INFO - stage3_gradient_single_runtime: 0.006316184997558594
2023-09-28 23:31:08,819 - utils - INFO - 1, epoch: 1510, all client loss: [0.5182338953018188, 0.45928335189819336], all pred client disparities: [0.010079652070999146, 0.0024670809507369995], all client disparities: [0.0032608509063720703, 0.008474290370941162], all client accs: [0.7457627654075623, 0.7918688654899597],alphas:tensor([0.5877, 0.4123, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:09,059 - utils - INFO - stage3_gradient_single_runtime: 0.006337642669677734
2023-09-28 23:31:09,066 - utils - INFO - 1, epoch: 1511, all client loss: [0.5182018280029297, 0.45904409885406494], all pred client disparities: [0.010152041912078857, 0.0023953914642333984], all client disparities: [0.0032608509063720703, 0.008035749197006226], all client accs: [0.7457627654075623, 0.792397677898407],alphas:tensor([0.6755, 0.0000, 0.3245, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:09,304 - utils - INFO - stage3_gradient_single_runtime: 0.006385087966918945
2023-09-28 23:31:09,308 - utils - INFO - 1, epoch: 1512, all client loss: [0.5181697010993958, 0.4590383768081665], all pred client disparities: [0.01014477014541626, 0.0024026036262512207], all client disparities: [0.0032608509063720703, 0.008035749197006226], all client accs: [0.7457627654075623, 0.792397677898407],alphas:tensor([0.6760, 0.0000, 0.3240, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:09,559 - utils - INFO - stage3_gradient_single_runtime: 0.006836891174316406
2023-09-28 23:31:09,563 - utils - INFO - 1, epoch: 1513, all client loss: [0.5181377530097961, 0.45903250575065613], all pred client disparities: [0.010137617588043213, 0.0024096667766571045], all client disparities: [0.0032608509063720703, 0.007315292954444885], all client accs: [0.7457627654075623, 0.7923665642738342],alphas:tensor([0.6765, 0.0000, 0.3235, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:09,800 - utils - INFO - stage3_gradient_single_runtime: 0.00627446174621582
2023-09-28 23:31:09,805 - utils - INFO - 1, epoch: 1514, all client loss: [0.5181058645248413, 0.4590265154838562], all pred client disparities: [0.010130643844604492, 0.0024165958166122437], all client disparities: [0.0032608509063720703, 0.007315292954444885], all client accs: [0.7457627654075623, 0.7923665642738342],alphas:tensor([0.6771, 0.0000, 0.3229, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:10,036 - utils - INFO - stage3_gradient_single_runtime: 0.0062906742095947266
2023-09-28 23:31:10,041 - utils - INFO - 1, epoch: 1515, all client loss: [0.5180739760398865, 0.45902034640312195], all pred client disparities: [0.01012393832206726, 0.002423405647277832], all client disparities: [0.0032608509063720703, 0.006594836711883545], all client accs: [0.7457627654075623, 0.792397677898407],alphas:tensor([0.6776, 0.0000, 0.3224, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:10,277 - utils - INFO - stage3_gradient_single_runtime: 0.006395101547241211
2023-09-28 23:31:10,283 - utils - INFO - 1, epoch: 1516, all client loss: [0.5180422067642212, 0.45901402831077576], all pred client disparities: [0.010117143392562866, 0.0024300217628479004], all client disparities: [0.0032608509063720703, 0.006521746050566435], all client accs: [0.7457627654075623, 0.7923665642738342],alphas:tensor([6.7810e-01, 4.5647e-17, 3.2190e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:10,522 - utils - INFO - stage3_gradient_single_runtime: 0.0062639713287353516
2023-09-28 23:31:10,529 - utils - INFO - 1, epoch: 1517, all client loss: [0.5180104970932007, 0.4590075612068176], all pred client disparities: [0.010110616683959961, 0.0024365782737731934], all client disparities: [0.0032608509063720703, 0.006594836711883545], all client accs: [0.7457627654075623, 0.7923665642738342],alphas:tensor([0.5899, 0.4101, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:10,760 - utils - INFO - stage3_gradient_single_runtime: 0.006429910659790039
2023-09-28 23:31:10,764 - utils - INFO - 1, epoch: 1518, all client loss: [0.5179790258407593, 0.45877039432525635], all pred client disparities: [0.010182946920394897, 0.00236491858959198], all client disparities: [0.0032608509063720703, 0.006521746050566435], all client accs: [0.7457627654075623, 0.7923665642738342],alphas:tensor([0.6768, 0.0000, 0.3232, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:11,001 - utils - INFO - stage3_gradient_single_runtime: 0.006226778030395508
2023-09-28 23:31:11,007 - utils - INFO - 1, epoch: 1519, all client loss: [0.5179473161697388, 0.4587652385234833], all pred client disparities: [0.010175555944442749, 0.0023722797632217407], all client disparities: [0.0032608509063720703, 0.006594836711883545], all client accs: [0.7457627654075623, 0.7923665642738342],alphas:tensor([0.6774, 0.0000, 0.3226, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:11,090 - utils - INFO - valid: True, epoch: 1519, loss: [0.577817976474762, 0.45806798338890076], accuracy: [0.7348066568374634, 0.7936645746231079], mean_accuracy:0.7642356157302856,variance_accuracy:0.029428958892822266, disparity: [0.004545450210571289, 0.003936827182769775], mean_disparity:0.004241138696670532,variance_disparity:0.00030431151390075684, pred_disparity: [0.004247575998306274, 0.0016820579767227173]
2023-09-28 23:31:11,215 - utils - INFO - global_valid: True, epoch: 1519,  global_loss: 0.4593992829322815, global_accuracy: 0.8214239265187817,  global_disparity:0.0004161149263381958, global_pred_disparity: 0.0017222315073013306,
2023-09-28 23:31:11,482 - utils - INFO - stage3_gradient_single_runtime: 0.0063779354095458984
2023-09-28 23:31:11,484 - utils - INFO - 1, epoch: 1520, all client loss: [0.5179156064987183, 0.4587599039077759], all pred client disparities: [0.010168403387069702, 0.0023794025182724], all client disparities: [0.0032608509063720703, 0.006594836711883545], all client accs: [0.7457627654075623, 0.7923665642738342],alphas:tensor([0.6779, 0.0000, 0.3221, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:11,725 - utils - INFO - stage3_gradient_single_runtime: 0.00630950927734375
2023-09-28 23:31:11,730 - utils - INFO - 1, epoch: 1521, all client loss: [0.5178840756416321, 0.4587544798851013], all pred client disparities: [0.010161340236663818, 0.00238645076751709], all client disparities: [0.0032608509063720703, 0.006594836711883545], all client accs: [0.7457627654075623, 0.7923665642738342],alphas:tensor([0.6784, 0.0000, 0.3216, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:12,006 - utils - INFO - stage3_gradient_single_runtime: 0.006311178207397461
2023-09-28 23:31:12,009 - utils - INFO - 1, epoch: 1522, all client loss: [0.5178524851799011, 0.45874881744384766], all pred client disparities: [0.010154396295547485, 0.0023933202028274536], all client disparities: [0.0032608509063720703, 0.005874365568161011], all client accs: [0.7457627654075623, 0.792397677898407],alphas:tensor([0.6790, 0.0000, 0.3210, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:12,244 - utils - INFO - stage3_gradient_single_runtime: 0.006331205368041992
2023-09-28 23:31:12,248 - utils - INFO - 1, epoch: 1523, all client loss: [0.5178210735321045, 0.4587430953979492], all pred client disparities: [0.010147631168365479, 0.0024000555276870728], all client disparities: [0.0032608509063720703, 0.005874365568161011], all client accs: [0.7457627654075623, 0.792397677898407],alphas:tensor([6.7948e-01, 4.5404e-17, 3.2052e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:12,539 - utils - INFO - stage3_gradient_single_runtime: 0.006309032440185547
2023-09-28 23:31:12,542 - utils - INFO - 1, epoch: 1524, all client loss: [0.5177896022796631, 0.45873716473579407], all pred client disparities: [0.010140955448150635, 0.0024066567420959473], all client disparities: [0.0014492571353912354, 0.005874365568161011], all client accs: [0.7481840252876282, 0.792397677898407],alphas:tensor([0.6800, 0.0000, 0.3200, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:12,773 - utils - INFO - stage3_gradient_single_runtime: 0.006277561187744141
2023-09-28 23:31:12,778 - utils - INFO - 1, epoch: 1525, all client loss: [0.517758309841156, 0.458731085062027], all pred client disparities: [0.01013454794883728, 0.002413138747215271], all client disparities: [0.0014492571353912354, 0.005874365568161011], all client accs: [0.7481840252876282, 0.7923665642738342],alphas:tensor([ 6.8050e-01, -9.0350e-17,  3.1950e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:13,036 - utils - INFO - stage3_gradient_single_runtime: 0.006340980529785156
2023-09-28 23:31:13,040 - utils - INFO - 1, epoch: 1526, all client loss: [0.5177269577980042, 0.4587249159812927], all pred client disparities: [0.010128140449523926, 0.00241948664188385], all client disparities: [0.0014492571353912354, 0.005874365568161011], all client accs: [0.7481840252876282, 0.7923665642738342],alphas:tensor([0.6810, 0.0000, 0.3190, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:13,272 - utils - INFO - stage3_gradient_single_runtime: 0.006229877471923828
2023-09-28 23:31:13,275 - utils - INFO - 1, epoch: 1527, all client loss: [0.5176957845687866, 0.45871856808662415], all pred client disparities: [0.01012188196182251, 0.002425670623779297], all client disparities: [0.0014492571353912354, 0.006093636155128479], all client accs: [0.7481840252876282, 0.7922109961509705],alphas:tensor([0.5951, 0.4049, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:13,505 - utils - INFO - stage3_gradient_single_runtime: 0.006270408630371094
2023-09-28 23:31:13,508 - utils - INFO - 1, epoch: 1528, all client loss: [0.5176647305488586, 0.45848461985588074], all pred client disparities: [0.010193496942520142, 0.0023547112941741943], all client disparities: [0.0014492571353912354, 0.005874365568161011], all client accs: [0.7481840252876282, 0.792397677898407],alphas:tensor([6.7973e-01, 4.5412e-17, 3.2027e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:13,741 - utils - INFO - stage3_gradient_single_runtime: 0.0063190460205078125
2023-09-28 23:31:13,746 - utils - INFO - 1, epoch: 1529, all client loss: [0.5176334381103516, 0.4584796130657196], all pred client disparities: [0.010186433792114258, 0.0023617595434188843], all client disparities: [0.0014492571353912354, 0.005874365568161011], all client accs: [0.7481840252876282, 0.7923665642738342],alphas:tensor([ 6.8025e-01, -9.0587e-17,  3.1975e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:13,978 - utils - INFO - stage3_gradient_single_runtime: 0.0062863826751708984
2023-09-28 23:31:13,983 - utils - INFO - 1, epoch: 1530, all client loss: [0.5176023244857788, 0.45847445726394653], all pred client disparities: [0.0101795494556427, 0.0023685991764068604], all client disparities: [0.0014492571353912354, 0.005874365568161011], all client accs: [0.7481840252876282, 0.7923665642738342],alphas:tensor([6.8076e-01, 4.5177e-17, 3.1924e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:14,215 - utils - INFO - stage3_gradient_single_runtime: 0.006293058395385742
2023-09-28 23:31:14,220 - utils - INFO - 1, epoch: 1531, all client loss: [0.5175711512565613, 0.4584691524505615], all pred client disparities: [0.010172814130783081, 0.002375274896621704], all client disparities: [0.0014492571353912354, 0.005728185176849365], all client accs: [0.7481840252876282, 0.7923043370246887],alphas:tensor([0.6813, 0.0000, 0.3187, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:14,453 - utils - INFO - stage3_gradient_single_runtime: 0.006355762481689453
2023-09-28 23:31:14,458 - utils - INFO - 1, epoch: 1532, all client loss: [0.5175401568412781, 0.4584636688232422], all pred client disparities: [0.010166257619857788, 0.002381846308708191], all client disparities: [0.0014492571353912354, 0.005728185176849365], all client accs: [0.7481840252876282, 0.7923043370246887],alphas:tensor([0.6818, 0.0000, 0.3182, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:14,644 - utils - INFO - stage3_gradient_single_runtime: 0.006384849548339844
2023-09-28 23:31:14,646 - utils - INFO - 1, epoch: 1533, all client loss: [0.5175090432167053, 0.4584580957889557], all pred client disparities: [0.010159730911254883, 0.002388313412666321], all client disparities: [0.0014492571353912354, 0.0059474557638168335], all client accs: [0.7481840252876282, 0.7920866012573242],alphas:tensor([0.6823, 0.0000, 0.3177, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:14,871 - utils - INFO - stage3_gradient_single_runtime: 0.0062563419342041016
2023-09-28 23:31:14,875 - utils - INFO - 1, epoch: 1534, all client loss: [0.5174781680107117, 0.45845234394073486], all pred client disparities: [0.010153383016586304, 0.0023946017026901245], all client disparities: [0.0014492571353912354, 0.0059474557638168335], all client accs: [0.7481840252876282, 0.7920866012573242],alphas:tensor([6.8277e-01, 4.4719e-17, 3.1723e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:15,105 - utils - INFO - stage3_gradient_single_runtime: 0.006835460662841797
2023-09-28 23:31:15,108 - utils - INFO - 1, epoch: 1535, all client loss: [0.517447292804718, 0.4584464430809021], all pred client disparities: [0.010147243738174438, 0.0024007856845855713], all client disparities: [0.0014492571353912354, 0.005226999521255493], all client accs: [0.7481840252876282, 0.792117714881897],alphas:tensor([0.6833, 0.0000, 0.3167, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:15,336 - utils - INFO - stage3_gradient_single_runtime: 0.006509542465209961
2023-09-28 23:31:15,339 - utils - INFO - 1, epoch: 1536, all client loss: [0.5174164175987244, 0.45844048261642456], all pred client disparities: [0.010141104459762573, 0.0024068206548690796], all client disparities: [0.0014492571353912354, 0.005226999521255493], all client accs: [0.7481840252876282, 0.792117714881897],alphas:tensor([0.6837, 0.0000, 0.3163, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:15,585 - utils - INFO - stage3_gradient_single_runtime: 0.008046388626098633
2023-09-28 23:31:15,589 - utils - INFO - 1, epoch: 1537, all client loss: [0.5173856616020203, 0.45843440294265747], all pred client disparities: [0.01013520359992981, 0.002412751317024231], all client disparities: [0.0014492571353912354, 0.005226999521255493], all client accs: [0.7481840252876282, 0.7921488285064697],alphas:tensor([0.6002, 0.3998, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:15,868 - utils - INFO - stage3_gradient_single_runtime: 0.007081031799316406
2023-09-28 23:31:15,875 - utils - INFO - 1, epoch: 1538, all client loss: [0.5173550844192505, 0.4582035541534424], all pred client disparities: [0.010206073522567749, 0.002342507243156433], all client disparities: [0.0014492571353912354, 0.004934653639793396], all client accs: [0.7481840252876282, 0.7923043370246887],alphas:tensor([6.8246e-01, 4.4847e-17, 3.1754e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:16,112 - utils - INFO - stage3_gradient_single_runtime: 0.006322145462036133
2023-09-28 23:31:16,118 - utils - INFO - 1, epoch: 1539, all client loss: [0.5173242688179016, 0.4581987261772156], all pred client disparities: [0.010199278593063354, 0.0023492127656936646], all client disparities: [0.0014492571353912354, 0.004934653639793396], all client accs: [0.7481840252876282, 0.7923043370246887],alphas:tensor([0.6830, 0.0000, 0.3170, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:16,248 - utils - INFO - valid: True, epoch: 1539, loss: [0.5776258111000061, 0.4574764668941498], accuracy: [0.7348066568374634, 0.793602466583252], mean_accuracy:0.7642045617103577,variance_accuracy:0.029397904872894287, disparity: [0.004545450210571289, 0.0037889033555984497], mean_disparity:0.004167176783084869,variance_disparity:0.0003782734274864197, pred_disparity: [0.004341632127761841, 0.0017054975032806396]
2023-09-28 23:31:16,327 - utils - INFO - global_valid: True, epoch: 1539,  global_loss: 0.4588122069835663, global_accuracy: 0.8220088715088611,  global_disparity:0.00027285516262054443, global_pred_disparity: 0.0016899257898330688,
2023-09-28 23:31:16,567 - utils - INFO - stage3_gradient_single_runtime: 0.006270885467529297
2023-09-28 23:31:16,572 - utils - INFO - 1, epoch: 1540, all client loss: [0.5172935128211975, 0.4581937789916992], all pred client disparities: [0.010192692279815674, 0.0023557692766189575], all client disparities: [0.0014492571353912354, 0.00515390932559967], all client accs: [0.7481840252876282, 0.7921488285064697],alphas:tensor([ 6.8346e-01, -8.9232e-17,  3.1654e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:16,808 - utils - INFO - stage3_gradient_single_runtime: 0.0063402652740478516
2023-09-28 23:31:16,814 - utils - INFO - 1, epoch: 1541, all client loss: [0.5172628164291382, 0.4581886827945709], all pred client disparities: [0.01018628478050232, 0.0023622065782546997], all client disparities: [0.0014492571353912354, 0.00515390932559967], all client accs: [0.7481840252876282, 0.7921488285064697],alphas:tensor([6.8396e-01, 4.4502e-17, 3.1604e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:17,051 - utils - INFO - stage3_gradient_single_runtime: 0.0063207149505615234
2023-09-28 23:31:17,057 - utils - INFO - 1, epoch: 1542, all client loss: [0.5172321796417236, 0.4581834673881531], all pred client disparities: [0.01017993688583374, 0.0023684650659561157], all client disparities: [0.0014492571353912354, 0.00515390932559967], all client accs: [0.7481840252876282, 0.7921488285064697],alphas:tensor([0.6844, 0.0000, 0.3156, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:17,294 - utils - INFO - stage3_gradient_single_runtime: 0.006333112716674805
2023-09-28 23:31:17,301 - utils - INFO - 1, epoch: 1543, all client loss: [0.5172016620635986, 0.4581781029701233], all pred client disparities: [0.010173648595809937, 0.00237467885017395], all client disparities: [0.0014492571353912354, 0.00515390932559967], all client accs: [0.7481840252876282, 0.7921488285064697],alphas:tensor([0.6849, 0.0000, 0.3151, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:17,537 - utils - INFO - stage3_gradient_single_runtime: 0.006387472152709961
2023-09-28 23:31:17,543 - utils - INFO - 1, epoch: 1544, all client loss: [0.5171710848808289, 0.45817264914512634], all pred client disparities: [0.010167628526687622, 0.002380669116973877], all client disparities: [0.0014492571353912354, 0.00515390932559967], all client accs: [0.7481840252876282, 0.7921488285064697],alphas:tensor([0.6854, 0.0000, 0.3146, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:17,779 - utils - INFO - stage3_gradient_single_runtime: 0.0063359737396240234
2023-09-28 23:31:17,785 - utils - INFO - 1, epoch: 1545, all client loss: [0.5171406269073486, 0.45816704630851746], all pred client disparities: [0.010161757469177246, 0.002386614680290222], all client disparities: [0.0014492571353912354, 0.005007728934288025], all client accs: [0.7481840252876282, 0.7924909591674805],alphas:tensor([0.6859, 0.0000, 0.3141, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:18,014 - utils - INFO - stage3_gradient_single_runtime: 0.006306171417236328
2023-09-28 23:31:18,021 - utils - INFO - 1, epoch: 1546, all client loss: [0.517110288143158, 0.45816129446029663], all pred client disparities: [0.010155856609344482, 0.002392381429672241], all client disparities: [0.0014492571353912354, 0.005080819129943848], all client accs: [0.7481840252876282, 0.7924909591674805],alphas:tensor([ 6.8637e-01, -8.7894e-17,  3.1363e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:18,264 - utils - INFO - stage3_gradient_single_runtime: 0.0074694156646728516
2023-09-28 23:31:18,269 - utils - INFO - 1, epoch: 1547, all client loss: [0.5170798897743225, 0.45815548300743103], all pred client disparities: [0.01015019416809082, 0.0023980140686035156], all client disparities: [0.0014492571353912354, 0.005080819129943848], all client accs: [0.7481840252876282, 0.7924909591674805],alphas:tensor([0.6051, 0.3949, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:18,503 - utils - INFO - stage3_gradient_single_runtime: 0.007345438003540039
2023-09-28 23:31:18,509 - utils - INFO - 1, epoch: 1548, all client loss: [0.5170497894287109, 0.4579276442527771], all pred client disparities: [0.010220378637313843, 0.002328440546989441], all client disparities: [0.0014492571353912354, 0.005007728934288025], all client accs: [0.7481840252876282, 0.7924909591674805],alphas:tensor([6.8506e-01, 4.4299e-17, 3.1494e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:18,748 - utils - INFO - stage3_gradient_single_runtime: 0.007086277008056641
2023-09-28 23:31:18,754 - utils - INFO - 1, epoch: 1549, all client loss: [0.5170194506645203, 0.45792311429977417], all pred client disparities: [0.010213881731033325, 0.0023349225521087646], all client disparities: [0.0014492571353912354, 0.005007728934288025], all client accs: [0.7481840252876282, 0.7924909591674805],alphas:tensor([0.6856, 0.0000, 0.3144, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:18,999 - utils - INFO - stage3_gradient_single_runtime: 0.007241010665893555
2023-09-28 23:31:19,004 - utils - INFO - 1, epoch: 1550, all client loss: [0.5169890522956848, 0.4579184353351593], all pred client disparities: [0.010207533836364746, 0.0023412108421325684], all client disparities: [0.0014492571353912354, 0.005811706185340881], all client accs: [0.7481840252876282, 0.792708694934845],alphas:tensor([0.6860, 0.0000, 0.3140, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:19,242 - utils - INFO - stage3_gradient_single_runtime: 0.006416797637939453
2023-09-28 23:31:19,248 - utils - INFO - 1, epoch: 1551, all client loss: [0.5169587731361389, 0.45791366696357727], all pred client disparities: [0.010201364755630493, 0.002347409725189209], all client disparities: [0.0014492571353912354, 0.005811706185340881], all client accs: [0.7481840252876282, 0.792708694934845],alphas:tensor([0.6865, 0.0000, 0.3135, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:19,481 - utils - INFO - stage3_gradient_single_runtime: 0.0064661502838134766
2023-09-28 23:31:19,487 - utils - INFO - 1, epoch: 1552, all client loss: [0.5169286131858826, 0.4579087197780609], all pred client disparities: [0.010195225477218628, 0.0023534148931503296], all client disparities: [0.0014492571353912354, 0.005811706185340881], all client accs: [0.7481840252876282, 0.792708694934845],alphas:tensor([0.6870, 0.0000, 0.3130, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:19,724 - utils - INFO - stage3_gradient_single_runtime: 0.006466865539550781
2023-09-28 23:31:19,729 - utils - INFO - 1, epoch: 1553, all client loss: [0.5168983936309814, 0.4579036831855774], all pred client disparities: [0.010189324617385864, 0.002359345555305481], all client disparities: [0.0014492571353912354, 0.005811706185340881], all client accs: [0.7481840252876282, 0.792708694934845],alphas:tensor([0.6875, 0.0000, 0.3125, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:19,962 - utils - INFO - stage3_gradient_single_runtime: 0.0062944889068603516
2023-09-28 23:31:19,965 - utils - INFO - 1, epoch: 1554, all client loss: [0.5168682932853699, 0.4578985273838043], all pred client disparities: [0.010183513164520264, 0.0023651123046875], all client disparities: [0.0014492571353912354, 0.005884796380996704], all client accs: [0.7481840252876282, 0.7927398085594177],alphas:tensor([0.6879, 0.0000, 0.3121, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:20,232 - utils - INFO - stage3_gradient_single_runtime: 0.0062754154205322266
2023-09-28 23:31:20,235 - utils - INFO - 1, epoch: 1555, all client loss: [0.5168382525444031, 0.4578932523727417], all pred client disparities: [0.010177761316299438, 0.002370774745941162], all client disparities: [0.0014492571353912354, 0.005884796380996704], all client accs: [0.7481840252876282, 0.792708694934845],alphas:tensor([0.6884, 0.0000, 0.3116, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:20,475 - utils - INFO - stage3_gradient_single_runtime: 0.010685443878173828
2023-09-28 23:31:20,479 - utils - INFO - 1, epoch: 1556, all client loss: [0.5168082118034363, 0.4578878879547119], all pred client disparities: [0.010172218084335327, 0.002376347780227661], all client disparities: [0.0014492571353912354, 0.005884796380996704], all client accs: [0.7481840252876282, 0.792708694934845],alphas:tensor([6.8888e-01, 4.3414e-17, 3.1112e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:20,724 - utils - INFO - stage3_gradient_single_runtime: 0.0063266754150390625
2023-09-28 23:31:20,727 - utils - INFO - 1, epoch: 1557, all client loss: [0.5167782306671143, 0.4578823447227478], all pred client disparities: [0.010166794061660767, 0.002381742000579834], all client disparities: [0.0014492571353912354, 0.005884796380996704], all client accs: [0.7481840252876282, 0.792708694934845],alphas:tensor([0.6893, 0.0000, 0.3107, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:20,961 - utils - INFO - stage3_gradient_single_runtime: 0.0063135623931884766
2023-09-28 23:31:20,966 - utils - INFO - 1, epoch: 1558, all client loss: [0.516748309135437, 0.4578767716884613], all pred client disparities: [0.01016145944595337, 0.0023870617151260376], all client disparities: [0.0014492571353912354, 0.005884796380996704], all client accs: [0.7481840252876282, 0.792677640914917],alphas:tensor([0.6898, 0.0000, 0.3102, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:21,195 - utils - INFO - stage3_gradient_single_runtime: 0.00636601448059082
2023-09-28 23:31:21,197 - utils - INFO - 1, epoch: 1559, all client loss: [0.5167185068130493, 0.45787104964256287], all pred client disparities: [0.010156184434890747, 0.0023922324180603027], all client disparities: [0.0014492571353912354, 0.005884796380996704], all client accs: [0.7481840252876282, 0.792677640914917],alphas:tensor([0.6118, 0.3882, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:21,279 - utils - INFO - valid: True, epoch: 1559, loss: [0.5774301886558533, 0.45690491795539856], accuracy: [0.7348066568374634, 0.7940372824668884], mean_accuracy:0.7644219696521759,variance_accuracy:0.029615312814712524, disparity: [0.004545450210571289, 0.002914339303970337], mean_disparity:0.003729894757270813,variance_disparity:0.0008155554533004761, pred_disparity: [0.004417538642883301, 0.0016971677541732788]
2023-09-28 23:31:21,406 - utils - INFO - global_valid: True, epoch: 1559,  global_loss: 0.45824483036994934, global_accuracy: 0.8225624730658707,  global_disparity:0.0005585700273513794, global_pred_disparity: 0.0016871839761734009,
2023-09-28 23:31:21,638 - utils - INFO - stage3_gradient_single_runtime: 0.006438255310058594
2023-09-28 23:31:21,643 - utils - INFO - 1, epoch: 1560, all client loss: [0.5166887640953064, 0.45764681696891785], all pred client disparities: [0.010225296020507812, 0.002323716878890991], all client disparities: [0.0014492571353912354, 0.00638599693775177], all client accs: [0.7481840252876282, 0.7926465272903442],alphas:tensor([6.8845e-01, 8.7110e-17, 3.1155e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:21,876 - utils - INFO - stage3_gradient_single_runtime: 0.006335258483886719
2023-09-28 23:31:21,881 - utils - INFO - 1, epoch: 1561, all client loss: [0.5166588425636292, 0.45764240622520447], all pred client disparities: [0.010219275951385498, 0.002329692244529724], all client disparities: [0.0014492571353912354, 0.00638599693775177], all client accs: [0.7481840252876282, 0.7926465272903442],alphas:tensor([6.8893e-01, 4.3444e-17, 3.1107e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:22,113 - utils - INFO - stage3_gradient_single_runtime: 0.006303310394287109
2023-09-28 23:31:22,118 - utils - INFO - 1, epoch: 1562, all client loss: [0.5166289806365967, 0.45763787627220154], all pred client disparities: [0.01021343469619751, 0.0023355036973953247], all client disparities: [0.0014492571353912354, 0.00638599693775177], all client accs: [0.7481840252876282, 0.7926154136657715],alphas:tensor([0.6894, 0.0000, 0.3106, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:22,353 - utils - INFO - stage3_gradient_single_runtime: 0.006298542022705078
2023-09-28 23:31:22,359 - utils - INFO - 1, epoch: 1563, all client loss: [0.5165992379188538, 0.45763325691223145], all pred client disparities: [0.010207653045654297, 0.00234125554561615], all client disparities: [0.0014492571353912354, 0.00638599693775177], all client accs: [0.7481840252876282, 0.792553186416626],alphas:tensor([6.8986e-01, 4.3225e-17, 3.1014e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:22,593 - utils - INFO - stage3_gradient_single_runtime: 0.006409168243408203
2023-09-28 23:31:22,598 - utils - INFO - 1, epoch: 1564, all client loss: [0.5165694355964661, 0.4576284885406494], all pred client disparities: [0.010202080011367798, 0.002346828579902649], all client disparities: [0.0014492571353912354, 0.005738615989685059], all client accs: [0.7481840252876282, 0.7926154136657715],alphas:tensor([0.6903, 0.0000, 0.3097, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:22,834 - utils - INFO - stage3_gradient_single_runtime: 0.006407260894775391
2023-09-28 23:31:22,839 - utils - INFO - 1, epoch: 1565, all client loss: [0.5165398120880127, 0.4576236605644226], all pred client disparities: [0.010196566581726074, 0.0023522675037384033], all client disparities: [0.0014492571353912354, 0.005738615989685059], all client accs: [0.7481840252876282, 0.7926154136657715],alphas:tensor([0.6908, 0.0000, 0.3092, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:23,077 - utils - INFO - stage3_gradient_single_runtime: 0.006350994110107422
2023-09-28 23:31:23,082 - utils - INFO - 1, epoch: 1566, all client loss: [0.5165101289749146, 0.45761868357658386], all pred client disparities: [0.010191231966018677, 0.0023576468229293823], all client disparities: [0.0014492571353912354, 0.005738615989685059], all client accs: [0.7481840252876282, 0.7926154136657715],alphas:tensor([0.6912, 0.0000, 0.3088, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:23,318 - utils - INFO - stage3_gradient_single_runtime: 0.006254673004150391
2023-09-28 23:31:23,323 - utils - INFO - 1, epoch: 1567, all client loss: [0.516480565071106, 0.45761358737945557], all pred client disparities: [0.010185927152633667, 0.002362847328186035], all client disparities: [0.0014492571353912354, 0.005738615989685059], all client accs: [0.7481840252876282, 0.7926154136657715],alphas:tensor([0.6917, 0.0000, 0.3083, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:23,554 - utils - INFO - stage3_gradient_single_runtime: 0.006312847137451172
2023-09-28 23:31:23,559 - utils - INFO - 1, epoch: 1568, all client loss: [0.5164510011672974, 0.4576084017753601], all pred client disparities: [0.010180741548538208, 0.0023679733276367188], all client disparities: [0.0014492571353912354, 0.005738615989685059], all client accs: [0.7481840252876282, 0.792553186416626],alphas:tensor([0.6921, 0.0000, 0.3079, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:23,788 - utils - INFO - stage3_gradient_single_runtime: 0.006382942199707031
2023-09-28 23:31:23,793 - utils - INFO - 1, epoch: 1569, all client loss: [0.5164214968681335, 0.4576031565666199], all pred client disparities: [0.01017579436302185, 0.002372950315475464], all client disparities: [0.0014492571353912354, 0.00566554069519043], all client accs: [0.7481840252876282, 0.7925220727920532],alphas:tensor([6.9258e-01, 4.2591e-17, 3.0742e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:24,024 - utils - INFO - stage3_gradient_single_runtime: 0.006375551223754883
2023-09-28 23:31:24,029 - utils - INFO - 1, epoch: 1570, all client loss: [0.516391932964325, 0.4575977921485901], all pred client disparities: [0.010170817375183105, 0.0023778676986694336], all client disparities: [0.0014492571353912354, 0.00566554069519043], all client accs: [0.7481840252876282, 0.7924909591674805],alphas:tensor([0.6930, 0.0000, 0.3070, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:24,259 - utils - INFO - stage3_gradient_single_runtime: 0.006300926208496094
2023-09-28 23:31:24,264 - utils - INFO - 1, epoch: 1571, all client loss: [0.5163625478744507, 0.45759230852127075], all pred client disparities: [0.010165989398956299, 0.002382636070251465], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7924909591674805],alphas:tensor([0.6182, 0.3818, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:24,494 - utils - INFO - stage3_gradient_single_runtime: 0.006311655044555664
2023-09-28 23:31:24,499 - utils - INFO - 1, epoch: 1572, all client loss: [0.516333281993866, 0.4573715329170227], all pred client disparities: [0.010234087705612183, 0.0023150891065597534], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7925843000411987],alphas:tensor([0.6917, 0.0000, 0.3083, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:24,728 - utils - INFO - stage3_gradient_single_runtime: 0.0063440799713134766
2023-09-28 23:31:24,733 - utils - INFO - 1, epoch: 1573, all client loss: [0.5163038969039917, 0.45736733078956604], all pred client disparities: [0.010228514671325684, 0.0023206472396850586], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([6.9211e-01, 4.2735e-17, 3.0789e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:24,967 - utils - INFO - stage3_gradient_single_runtime: 0.006326913833618164
2023-09-28 23:31:24,972 - utils - INFO - 1, epoch: 1574, all client loss: [0.5162744522094727, 0.457363098859787], all pred client disparities: [0.010223060846328735, 0.002326086163520813], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([0.6926, 0.0000, 0.3074, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:25,203 - utils - INFO - stage3_gradient_single_runtime: 0.006241798400878906
2023-09-28 23:31:25,207 - utils - INFO - 1, epoch: 1575, all client loss: [0.5162451267242432, 0.4573587477207184], all pred client disparities: [0.01021769642829895, 0.0023314207792282104], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([ 6.9302e-01, -8.5043e-17,  3.0698e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:25,437 - utils - INFO - stage3_gradient_single_runtime: 0.006344795227050781
2023-09-28 23:31:25,442 - utils - INFO - 1, epoch: 1576, all client loss: [0.5162158012390137, 0.45735424757003784], all pred client disparities: [0.010212510824203491, 0.0023365765810012817], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([0.6935, 0.0000, 0.3065, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:25,675 - utils - INFO - stage3_gradient_single_runtime: 0.0063402652740478516
2023-09-28 23:31:25,680 - utils - INFO - 1, epoch: 1577, all client loss: [0.516186535358429, 0.4573496878147125], all pred client disparities: [0.01020735502243042, 0.00234164297580719], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([ 6.9391e-01, -8.4624e-17,  3.0609e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:25,912 - utils - INFO - stage3_gradient_single_runtime: 0.006290435791015625
2023-09-28 23:31:25,917 - utils - INFO - 1, epoch: 1578, all client loss: [0.516157329082489, 0.45734503865242004], all pred client disparities: [0.010202348232269287, 0.002346605062484741], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([6.9435e-01, 4.2208e-17, 3.0565e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:26,151 - utils - INFO - stage3_gradient_single_runtime: 0.006340742111206055
2023-09-28 23:31:26,156 - utils - INFO - 1, epoch: 1579, all client loss: [0.5161281824111938, 0.457340270280838], all pred client disparities: [0.010197460651397705, 0.0023514628410339355], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7925843000411987],alphas:tensor([0.6948, 0.0000, 0.3052, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:26,237 - utils - INFO - valid: True, epoch: 1579, loss: [0.5773247480392456, 0.4565768539905548], accuracy: [0.7348066568374634, 0.7940993905067444], mean_accuracy:0.7644530236721039,variance_accuracy:0.029646366834640503, disparity: [0.004545450210571289, 0.0027664154767990112], mean_disparity:0.00365593284368515,variance_disparity:0.0008895173668861389, pred_disparity: [0.004429876804351807, 0.0017066597938537598]
2023-09-28 23:31:26,363 - utils - INFO - global_valid: True, epoch: 1579,  global_loss: 0.45791923999786377, global_accuracy: 0.8230284949234721,  global_disparity:0.0007018446922302246, global_pred_disparity: 0.001661449670791626,
2023-09-28 23:31:26,597 - utils - INFO - stage3_gradient_single_runtime: 0.006333351135253906
2023-09-28 23:31:26,602 - utils - INFO - 1, epoch: 1580, all client loss: [0.5160989761352539, 0.4573354423046112], all pred client disparities: [0.010192722082138062, 0.0023561865091323853], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7925843000411987],alphas:tensor([0.6952, 0.0000, 0.3048, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:26,837 - utils - INFO - stage3_gradient_single_runtime: 0.006283760070800781
2023-09-28 23:31:26,841 - utils - INFO - 1, epoch: 1581, all client loss: [0.5160698294639587, 0.45733052492141724], all pred client disparities: [0.010188043117523193, 0.002360820770263672], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7925843000411987],alphas:tensor([ 6.9565e-01, -8.3805e-17,  3.0435e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:27,077 - utils - INFO - stage3_gradient_single_runtime: 0.00633549690246582
2023-09-28 23:31:27,083 - utils - INFO - 1, epoch: 1582, all client loss: [0.5160408020019531, 0.4573254883289337], all pred client disparities: [0.010183513164520264, 0.00236530601978302], all client disparities: [0.0032608509063720703, 0.005738615989685059], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([6.9608e-01, 4.1802e-17, 3.0392e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:27,325 - utils - INFO - stage3_gradient_single_runtime: 0.006316184997558594
2023-09-28 23:31:27,330 - utils - INFO - 1, epoch: 1583, all client loss: [0.5160117745399475, 0.45732036232948303], all pred client disparities: [0.010179072618484497, 0.0023697614669799805], all client disparities: [0.0032608509063720703, 0.005811706185340881], all client accs: [0.7506053447723389, 0.792553186416626],alphas:tensor([ 6.9650e-01, -8.3405e-17,  3.0350e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:27,514 - utils - INFO - stage3_gradient_single_runtime: 0.0062863826751708984
2023-09-28 23:31:27,520 - utils - INFO - 1, epoch: 1584, all client loss: [0.5159828066825867, 0.4573151767253876], all pred client disparities: [0.010174721479415894, 0.0023740530014038086], all client disparities: [0.0032608509063720703, 0.005811706185340881], all client accs: [0.7506053447723389, 0.792553186416626],alphas:tensor([0.6253, 0.3747, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:27,760 - utils - INFO - stage3_gradient_single_runtime: 0.006315469741821289
2023-09-28 23:31:27,766 - utils - INFO - 1, epoch: 1585, all client loss: [0.5159540176391602, 0.45709794759750366], all pred client disparities: [0.01024162769317627, 0.002307683229446411], all client disparities: [0.0032608509063720703, 0.005738615989685059], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([0.6951, 0.0000, 0.3049, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:28,021 - utils - INFO - stage3_gradient_single_runtime: 0.00629425048828125
2023-09-28 23:31:28,027 - utils - INFO - 1, epoch: 1586, all client loss: [0.5159250497817993, 0.45709413290023804], all pred client disparities: [0.010236501693725586, 0.002312764525413513], all client disparities: [0.0032608509063720703, 0.005738615989685059], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([ 6.9555e-01, -8.3908e-17,  3.0445e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:28,265 - utils - INFO - stage3_gradient_single_runtime: 0.006947040557861328
2023-09-28 23:31:28,271 - utils - INFO - 1, epoch: 1587, all client loss: [0.5158960819244385, 0.4570901095867157], all pred client disparities: [0.010231494903564453, 0.0023177266120910645], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7925843000411987],alphas:tensor([0.6960, 0.0000, 0.3040, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:28,512 - utils - INFO - stage3_gradient_single_runtime: 0.006799459457397461
2023-09-28 23:31:28,517 - utils - INFO - 1, epoch: 1588, all client loss: [0.5158671736717224, 0.45708605647087097], all pred client disparities: [0.010226547718048096, 0.0023225992918014526], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7925843000411987],alphas:tensor([0.6964, 0.0000, 0.3036, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:28,770 - utils - INFO - stage3_gradient_single_runtime: 0.006353139877319336
2023-09-28 23:31:28,775 - utils - INFO - 1, epoch: 1589, all client loss: [0.5158383250236511, 0.4570819139480591], all pred client disparities: [0.010221868753433228, 0.0023273080587387085], all client disparities: [0.0032608509063720703, 0.00566554069519043], all client accs: [0.7506053447723389, 0.7925843000411987],alphas:tensor([0.6969, 0.0000, 0.3031, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:29,006 - utils - INFO - stage3_gradient_single_runtime: 0.006419658660888672
2023-09-28 23:31:29,012 - utils - INFO - 1, epoch: 1590, all client loss: [0.5158094763755798, 0.45707765221595764], all pred client disparities: [0.010217159986495972, 0.002331942319869995], all client disparities: [0.0032608509063720703, 0.005738615989685059], all client accs: [0.7506053447723389, 0.7925220727920532],alphas:tensor([0.6973, 0.0000, 0.3027, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:29,255 - utils - INFO - stage3_gradient_single_runtime: 0.006418704986572266
2023-09-28 23:31:29,261 - utils - INFO - 1, epoch: 1591, all client loss: [0.5157806873321533, 0.4570733308792114], all pred client disparities: [0.010212630033493042, 0.002336472272872925], all client disparities: [0.0032608509063720703, 0.005738615989685059], all client accs: [0.7506053447723389, 0.7924909591674805],alphas:tensor([6.9773e-01, 4.1443e-17, 3.0227e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:29,498 - utils - INFO - stage3_gradient_single_runtime: 0.006289005279541016
2023-09-28 23:31:29,503 - utils - INFO - 1, epoch: 1592, all client loss: [0.5157519578933716, 0.45706892013549805], all pred client disparities: [0.010208159685134888, 0.0023408830165863037], all client disparities: [0.0032608509063720703, 0.005675971508026123], all client accs: [0.7506053447723389, 0.792677640914917],alphas:tensor([6.9815e-01, 4.1343e-17, 3.0185e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:29,744 - utils - INFO - stage3_gradient_single_runtime: 0.0064961910247802734
2023-09-28 23:31:29,749 - utils - INFO - 1, epoch: 1593, all client loss: [0.5157232284545898, 0.4570644199848175], all pred client disparities: [0.010203808546066284, 0.0023451894521713257], all client disparities: [0.0032608509063720703, 0.005675971508026123], all client accs: [0.7506053447723389, 0.792677640914917],alphas:tensor([6.9857e-01, 4.1244e-17, 3.0143e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:29,993 - utils - INFO - stage3_gradient_single_runtime: 0.006266117095947266
2023-09-28 23:31:29,999 - utils - INFO - 1, epoch: 1594, all client loss: [0.5156945586204529, 0.4570598304271698], all pred client disparities: [0.010199576616287231, 0.0023493915796279907], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7926465272903442],alphas:tensor([0.6990, 0.0000, 0.3010, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:30,237 - utils - INFO - stage3_gradient_single_runtime: 0.0064868927001953125
2023-09-28 23:31:30,243 - utils - INFO - 1, epoch: 1595, all client loss: [0.5156658887863159, 0.4570551812648773], all pred client disparities: [0.010195404291152954, 0.0023535490036010742], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7926465272903442],alphas:tensor([0.6994, 0.0000, 0.3006, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:30,482 - utils - INFO - stage3_gradient_single_runtime: 0.006247520446777344
2023-09-28 23:31:30,488 - utils - INFO - 1, epoch: 1596, all client loss: [0.5156372785568237, 0.4570504426956177], all pred client disparities: [0.010191410779953003, 0.0023575276136398315], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7926465272903442],alphas:tensor([0.6998, 0.0000, 0.3002, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:30,730 - utils - INFO - stage3_gradient_single_runtime: 0.006951332092285156
2023-09-28 23:31:30,737 - utils - INFO - 1, epoch: 1597, all client loss: [0.5156087279319763, 0.45704567432403564], all pred client disparities: [0.010187417268753052, 0.0023614168167114258], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7926465272903442],alphas:tensor([0.7002, 0.0000, 0.2998, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:30,970 - utils - INFO - stage3_gradient_single_runtime: 0.006240367889404297
2023-09-28 23:31:30,973 - utils - INFO - 1, epoch: 1598, all client loss: [0.5155801177024841, 0.45704081654548645], all pred client disparities: [0.010183602571487427, 0.002365216612815857], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([7.0062e-01, 4.0760e-17, 2.9938e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:31,209 - utils - INFO - stage3_gradient_single_runtime: 0.006295919418334961
2023-09-28 23:31:31,214 - utils - INFO - 1, epoch: 1599, all client loss: [0.5155516862869263, 0.4570358693599701], all pred client disparities: [0.010179787874221802, 0.002368941903114319], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7926154136657715],alphas:tensor([0.6337, 0.3663, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:31,343 - utils - INFO - valid: True, epoch: 1599, loss: [0.5771176815032959, 0.4560408592224121], accuracy: [0.7348066568374634, 0.7940372824668884], mean_accuracy:0.7644219696521759,variance_accuracy:0.029615312814712524, disparity: [0.004545450210571289, 0.0020397603511810303], mean_disparity:0.0032926052808761597,variance_disparity:0.0012528449296951294, pred_disparity: [0.004484504461288452, 0.0016276687383651733]
2023-09-28 23:31:31,420 - utils - INFO - global_valid: True, epoch: 1599,  global_loss: 0.45738691091537476, global_accuracy: 0.8234788137001958,  global_disparity:0.0013899952173233032, global_pred_disparity: 0.0017252415418624878,
2023-09-28 23:31:31,654 - utils - INFO - stage3_gradient_single_runtime: 0.006272792816162109
2023-09-28 23:31:31,659 - utils - INFO - 1, epoch: 1600, all client loss: [0.515523374080658, 0.4568227231502533], all pred client disparities: [0.010245293378829956, 0.0023040175437927246], all client disparities: [0.0032608509063720703, 0.006250247359275818], all client accs: [0.7506053447723389, 0.792708694934845],alphas:tensor([6.9919e-01, 4.1118e-17, 3.0081e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:31,894 - utils - INFO - stage3_gradient_single_runtime: 0.006279468536376953
2023-09-28 23:31:31,899 - utils - INFO - 1, epoch: 1601, all client loss: [0.5154948234558105, 0.456819087266922], all pred client disparities: [0.010240733623504639, 0.002308472990989685], all client disparities: [0.0032608509063720703, 0.006250247359275818], all client accs: [0.7506053447723389, 0.792708694934845],alphas:tensor([0.6996, 0.0000, 0.3004, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:32,136 - utils - INFO - stage3_gradient_single_runtime: 0.00694584846496582
2023-09-28 23:31:32,141 - utils - INFO - 1, epoch: 1602, all client loss: [0.5154663920402527, 0.4568153917789459], all pred client disparities: [0.010236352682113647, 0.0023128539323806763], all client disparities: [0.0032608509063720703, 0.006250247359275818], all client accs: [0.7506053447723389, 0.792708694934845],alphas:tensor([7.0004e-01, 4.0918e-17, 2.9996e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:32,392 - utils - INFO - stage3_gradient_single_runtime: 0.006330728530883789
2023-09-28 23:31:32,397 - utils - INFO - 1, epoch: 1603, all client loss: [0.5154379606246948, 0.4568116068840027], all pred client disparities: [0.010232031345367432, 0.0023171156644821167], all client disparities: [0.0032608509063720703, 0.006396427750587463], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7005, 0.0000, 0.2995, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:32,638 - utils - INFO - stage3_gradient_single_runtime: 0.006302595138549805
2023-09-28 23:31:32,643 - utils - INFO - 1, epoch: 1604, all client loss: [0.515409529209137, 0.4568077623844147], all pred client disparities: [0.010227799415588379, 0.002321287989616394], all client disparities: [0.0032608509063720703, 0.006396427750587463], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([ 7.0088e-01, -8.1442e-17,  2.9912e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:32,874 - utils - INFO - stage3_gradient_single_runtime: 0.006277799606323242
2023-09-28 23:31:32,879 - utils - INFO - 1, epoch: 1605, all client loss: [0.5153812170028687, 0.4568038284778595], all pred client disparities: [0.010223716497421265, 0.0023253560066223145], all client disparities: [0.0032608509063720703, 0.006323337554931641], all client accs: [0.7506053447723389, 0.792708694934845],alphas:tensor([7.0130e-01, 4.0623e-17, 2.9870e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:33,108 - utils - INFO - stage3_gradient_single_runtime: 0.006270647048950195
2023-09-28 23:31:33,113 - utils - INFO - 1, epoch: 1606, all client loss: [0.5153529047966003, 0.45679980516433716], all pred client disparities: [0.010219752788543701, 0.002329319715499878], all client disparities: [0.0032608509063720703, 0.006323337554931641], all client accs: [0.7506053447723389, 0.792708694934845],alphas:tensor([ 7.0171e-01, -8.1053e-17,  2.9829e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:33,341 - utils - INFO - stage3_gradient_single_runtime: 0.0062940120697021484
2023-09-28 23:31:33,346 - utils - INFO - 1, epoch: 1607, all client loss: [0.515324592590332, 0.45679575204849243], all pred client disparities: [0.010215908288955688, 0.0023331493139266968], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7021, 0.0000, 0.2979, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:33,589 - utils - INFO - stage3_gradient_single_runtime: 0.006299257278442383
2023-09-28 23:31:33,594 - utils - INFO - 1, epoch: 1608, all client loss: [0.5152963399887085, 0.45679160952568054], all pred client disparities: [0.010212063789367676, 0.002336934208869934], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([7.0252e-01, 4.0335e-17, 2.9748e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:33,875 - utils - INFO - stage3_gradient_single_runtime: 0.006592273712158203
2023-09-28 23:31:33,881 - utils - INFO - 1, epoch: 1609, all client loss: [0.5152681469917297, 0.4567873775959015], all pred client disparities: [0.010208368301391602, 0.0023405998945236206], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([7.0292e-01, 4.0241e-17, 2.9708e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:34,113 - utils - INFO - stage3_gradient_single_runtime: 0.0062770843505859375
2023-09-28 23:31:34,120 - utils - INFO - 1, epoch: 1610, all client loss: [0.515239953994751, 0.45678311586380005], all pred client disparities: [0.010204732418060303, 0.00234416127204895], all client disparities: [0.0032608509063720703, 0.005675971508026123], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([7.0332e-01, 4.0147e-17, 2.9668e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:34,357 - utils - INFO - stage3_gradient_single_runtime: 0.006257534027099609
2023-09-28 23:31:34,362 - utils - INFO - 1, epoch: 1611, all client loss: [0.5152117609977722, 0.45677879452705383], all pred client disparities: [0.010201215744018555, 0.0023476630449295044], all client disparities: [0.0032608509063720703, 0.005675971508026123], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7037, 0.0000, 0.2963, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:34,597 - utils - INFO - stage3_gradient_single_runtime: 0.009670019149780273
2023-09-28 23:31:34,603 - utils - INFO - 1, epoch: 1612, all client loss: [0.515183687210083, 0.45677438378334045], all pred client disparities: [0.010197848081588745, 0.00235101580619812], all client disparities: [0.0032608509063720703, 0.005675971508026123], all client accs: [0.7506053447723389, 0.792708694934845],alphas:tensor([0.7041, 0.0000, 0.2959, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:34,868 - utils - INFO - stage3_gradient_single_runtime: 0.006342411041259766
2023-09-28 23:31:34,874 - utils - INFO - 1, epoch: 1613, all client loss: [0.515155553817749, 0.4567699432373047], all pred client disparities: [0.010194450616836548, 0.002354338765144348], all client disparities: [0.0032608509063720703, 0.005675971508026123], all client accs: [0.7506053447723389, 0.792708694934845],alphas:tensor([0.7045, 0.0000, 0.2955, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:35,167 - utils - INFO - stage3_gradient_single_runtime: 0.009690284729003906
2023-09-28 23:31:35,172 - utils - INFO - 1, epoch: 1614, all client loss: [0.5151274800300598, 0.45676544308662415], all pred client disparities: [0.010191202163696289, 0.0023575574159622192], all client disparities: [0.0032608509063720703, 0.005749046802520752], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7049, 0.0000, 0.2951, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:35,446 - utils - INFO - stage3_gradient_single_runtime: 0.006318330764770508
2023-09-28 23:31:35,451 - utils - INFO - 1, epoch: 1615, all client loss: [0.5150994658470154, 0.45676088333129883], all pred client disparities: [0.010188072919845581, 0.002360701560974121], all client disparities: [0.0032608509063720703, 0.005749046802520752], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.6425, 0.3575, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:35,731 - utils - INFO - stage3_gradient_single_runtime: 0.00981450080871582
2023-09-28 23:31:35,738 - utils - INFO - 1, epoch: 1616, all client loss: [0.5150716304779053, 0.45655179023742676], all pred client disparities: [0.010252028703689575, 0.00229722261428833], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7034, 0.0000, 0.2966, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:36,025 - utils - INFO - stage3_gradient_single_runtime: 0.009268045425415039
2023-09-28 23:31:36,030 - utils - INFO - 1, epoch: 1617, all client loss: [0.5150436162948608, 0.4565484821796417], all pred client disparities: [0.010248064994812012, 0.0023010969161987305], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7038, 0.0000, 0.2962, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:36,274 - utils - INFO - stage3_gradient_single_runtime: 0.006696939468383789
2023-09-28 23:31:36,279 - utils - INFO - 1, epoch: 1618, all client loss: [0.5150156021118164, 0.4565452039241791], all pred client disparities: [0.010244280099868774, 0.0023049116134643555], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7042, 0.0000, 0.2958, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:36,541 - utils - INFO - stage3_gradient_single_runtime: 0.006768703460693359
2023-09-28 23:31:36,546 - utils - INFO - 1, epoch: 1619, all client loss: [0.5149877071380615, 0.45654183626174927], all pred client disparities: [0.010240554809570312, 0.002308592200279236], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([7.0464e-01, 3.9852e-17, 2.9536e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:36,631 - utils - INFO - valid: True, epoch: 1619, loss: [0.5769981741905212, 0.4557407796382904], accuracy: [0.7348066568374634, 0.7939130663871765], mean_accuracy:0.76435986161232,variance_accuracy:0.029553204774856567, disparity: [0.004545450210571289, 0.0020397603511810303], mean_disparity:0.0032926052808761597,variance_disparity:0.0012528449296951294, pred_disparity: [0.004484385251998901, 0.0015566200017929077]
2023-09-28 23:31:36,788 - utils - INFO - global_valid: True, epoch: 1619,  global_loss: 0.4570888578891754, global_accuracy: 0.8238853373998248,  global_disparity:0.0013899952173233032, global_pred_disparity: 0.0017756819725036621,
2023-09-28 23:31:37,057 - utils - INFO - stage3_gradient_single_runtime: 0.0069005489349365234
2023-09-28 23:31:37,062 - utils - INFO - 1, epoch: 1620, all client loss: [0.5149597525596619, 0.4565383791923523], all pred client disparities: [0.010236948728561401, 0.0023121684789657593], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7050, 0.0000, 0.2950, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:37,334 - utils - INFO - stage3_gradient_single_runtime: 0.006761074066162109
2023-09-28 23:31:37,339 - utils - INFO - 1, epoch: 1621, all client loss: [0.514931857585907, 0.45653489232063293], all pred client disparities: [0.010233372449874878, 0.0023156702518463135], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([7.0545e-01, 3.9664e-17, 2.9455e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:37,604 - utils - INFO - stage3_gradient_single_runtime: 0.00657963752746582
2023-09-28 23:31:37,609 - utils - INFO - 1, epoch: 1622, all client loss: [0.5149040818214417, 0.4565313160419464], all pred client disparities: [0.01022997498512268, 0.002319052815437317], all client disparities: [0.0032608509063720703, 0.0056028813123703], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7058, 0.0000, 0.2942, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:37,869 - utils - INFO - stage3_gradient_single_runtime: 0.006613492965698242
2023-09-28 23:31:37,874 - utils - INFO - 1, epoch: 1623, all client loss: [0.5148762464523315, 0.4565276503562927], all pred client disparities: [0.010226637125015259, 0.002322375774383545], all client disparities: [0.0032608509063720703, 0.005675971508026123], all client accs: [0.7506053447723389, 0.7927709221839905],alphas:tensor([0.7062, 0.0000, 0.2938, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:38,140 - utils - INFO - stage3_gradient_single_runtime: 0.00668787956237793
2023-09-28 23:31:38,145 - utils - INFO - 1, epoch: 1624, all client loss: [0.5148484706878662, 0.45652398467063904], all pred client disparities: [0.010223329067230225, 0.0023256242275238037], all client disparities: [0.0032608509063720703, 0.004955500364303589], all client accs: [0.7506053447723389, 0.792833149433136],alphas:tensor([0.7066, 0.0000, 0.2934, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:38,455 - utils - INFO - stage3_gradient_single_runtime: 0.006647825241088867
2023-09-28 23:31:38,459 - utils - INFO - 1, epoch: 1625, all client loss: [0.5148206949234009, 0.4565202295780182], all pred client disparities: [0.010220199823379517, 0.00232870876789093], all client disparities: [0.0032608509063720703, 0.004955500364303589], all client accs: [0.7506053447723389, 0.792833149433136],alphas:tensor([7.0701e-01, 3.9297e-17, 2.9299e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:38,726 - utils - INFO - stage3_gradient_single_runtime: 0.0067920684814453125
2023-09-28 23:31:38,732 - utils - INFO - 1, epoch: 1626, all client loss: [0.5147929787635803, 0.45651644468307495], all pred client disparities: [0.010217100381851196, 0.002331763505935669], all client disparities: [0.0032608509063720703, 0.004882410168647766], all client accs: [0.7506053447723389, 0.792833149433136],alphas:tensor([0.7074, 0.0000, 0.2926, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:38,998 - utils - INFO - stage3_gradient_single_runtime: 0.006726503372192383
2023-09-28 23:31:39,003 - utils - INFO - 1, epoch: 1627, all client loss: [0.5147652626037598, 0.45651260018348694], all pred client disparities: [0.010214060544967651, 0.0023347139358520508], all client disparities: [0.0032608509063720703, 0.004955500364303589], all client accs: [0.7506053447723389, 0.7928642630577087],alphas:tensor([0.7078, 0.0000, 0.2922, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:39,265 - utils - INFO - stage3_gradient_single_runtime: 0.006676435470581055
2023-09-28 23:31:39,270 - utils - INFO - 1, epoch: 1628, all client loss: [0.5147375464439392, 0.45650869607925415], all pred client disparities: [0.010211139917373657, 0.0023375749588012695], all client disparities: [0.0032608509063720703, 0.004955500364303589], all client accs: [0.7506053447723389, 0.7928642630577087],alphas:tensor([7.0816e-01, 3.9029e-17, 2.9184e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:39,530 - utils - INFO - stage3_gradient_single_runtime: 0.006536722183227539
2023-09-28 23:31:39,534 - utils - INFO - 1, epoch: 1629, all client loss: [0.5147099494934082, 0.456504762172699], all pred client disparities: [0.010208338499069214, 0.0023403912782669067], all client disparities: [0.0032608509063720703, 0.004955500364303589], all client accs: [0.7506053447723389, 0.792833149433136],alphas:tensor([ 7.0853e-01, -7.7883e-17,  2.9147e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:39,786 - utils - INFO - stage3_gradient_single_runtime: 0.006711244583129883
2023-09-28 23:31:39,790 - utils - INFO - 1, epoch: 1630, all client loss: [0.5146823525428772, 0.456500768661499], all pred client disparities: [0.010205596685409546, 0.002343088388442993], all client disparities: [0.0032608509063720703, 0.004955500364303589], all client accs: [0.7506053447723389, 0.792833149433136],alphas:tensor([0.7089, 0.0000, 0.2911, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:40,046 - utils - INFO - stage3_gradient_single_runtime: 0.006649494171142578
2023-09-28 23:31:40,051 - utils - INFO - 1, epoch: 1631, all client loss: [0.5146547555923462, 0.4564967453479767], all pred client disparities: [0.010202914476394653, 0.0023457109928131104], all client disparities: [0.0032608509063720703, 0.004955500364303589], all client accs: [0.7506053447723389, 0.7927709221839905],alphas:tensor([0.7093, 0.0000, 0.2907, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:40,308 - utils - INFO - stage3_gradient_single_runtime: 0.0067064762115478516
2023-09-28 23:31:40,313 - utils - INFO - 1, epoch: 1632, all client loss: [0.5146271586418152, 0.45649266242980957], all pred client disparities: [0.010200411081314087, 0.0023482441902160645], all client disparities: [0.0032608509063720703, 0.0038069337606430054], all client accs: [0.7506053447723389, 0.7928020358085632],alphas:tensor([0.7096, 0.0000, 0.2904, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:40,578 - utils - INFO - stage3_gradient_single_runtime: 0.007687568664550781
2023-09-28 23:31:40,584 - utils - INFO - 1, epoch: 1633, all client loss: [0.514599621295929, 0.45648854970932007], all pred client disparities: [0.010197848081588745, 0.002350717782974243], all client disparities: [0.0032608509063720703, 0.0038069337606430054], all client accs: [0.7506053447723389, 0.7927709221839905],alphas:tensor([0.7100, 0.0000, 0.2900, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:40,873 - utils - INFO - stage3_gradient_single_runtime: 0.0072247982025146484
2023-09-28 23:31:40,879 - utils - INFO - 1, epoch: 1634, all client loss: [0.5145721435546875, 0.4564844071865082], all pred client disparities: [0.010195434093475342, 0.0023531317710876465], all client disparities: [0.0032608509063720703, 0.0038069337606430054], all client accs: [0.7506053447723389, 0.7927709221839905],alphas:tensor([0.7104, 0.0000, 0.2896, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:41,138 - utils - INFO - stage3_gradient_single_runtime: 0.0065305233001708984
2023-09-28 23:31:41,142 - utils - INFO - 1, epoch: 1635, all client loss: [0.5145447254180908, 0.4564802050590515], all pred client disparities: [0.010193109512329102, 0.002355441451072693], all client disparities: [0.0032608509063720703, 0.0038069337606430054], all client accs: [0.7506053447723389, 0.792708694934845],alphas:tensor([0.6539, 0.3461, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:41,402 - utils - INFO - stage3_gradient_single_runtime: 0.0065155029296875
2023-09-28 23:31:41,407 - utils - INFO - 1, epoch: 1636, all client loss: [0.5145174264907837, 0.45627593994140625], all pred client disparities: [0.01025506854057312, 0.0022939294576644897], all client disparities: [0.0032608509063720703, 0.0038069337606430054], all client accs: [0.7506053447723389, 0.7928020358085632],alphas:tensor([0.7089, 0.0000, 0.2911, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:41,678 - utils - INFO - stage3_gradient_single_runtime: 0.00662684440612793
2023-09-28 23:31:41,682 - utils - INFO - 1, epoch: 1637, all client loss: [0.5144899487495422, 0.4562731087207794], all pred client disparities: [0.0102519690990448, 0.0022969990968704224], all client disparities: [0.0032608509063720703, 0.0038069337606430054], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7092, 0.0000, 0.2908, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:41,939 - utils - INFO - stage3_gradient_single_runtime: 0.0066967010498046875
2023-09-28 23:31:41,944 - utils - INFO - 1, epoch: 1638, all client loss: [0.5144625306129456, 0.4562701880931854], all pred client disparities: [0.010248959064483643, 0.002299964427947998], all client disparities: [0.0032608509063720703, 0.003086462616920471], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7096, 0.0000, 0.2904, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:42,210 - utils - INFO - stage3_gradient_single_runtime: 0.0068018436431884766
2023-09-28 23:31:42,215 - utils - INFO - 1, epoch: 1639, all client loss: [0.5144352316856384, 0.45626720786094666], all pred client disparities: [0.010246038436889648, 0.0023028403520584106], all client disparities: [0.0032608509063720703, 0.003159552812576294], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([ 7.1001e-01, -7.7211e-17,  2.8999e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:42,304 - utils - INFO - valid: True, epoch: 1639, loss: [0.57686847448349, 0.4554511308670044], accuracy: [0.7348066568374634, 0.7941614985466003], mean_accuracy:0.7644840776920319,variance_accuracy:0.02967742085456848, disparity: [0.004545450210571289, 0.002927333116531372], mean_disparity:0.0037363916635513306,variance_disparity:0.0008090585470199585, pred_disparity: [0.004484295845031738, 0.0014414787292480469]
2023-09-28 23:31:42,451 - utils - INFO - global_valid: True, epoch: 1639,  global_loss: 0.45680099725723267, global_accuracy: 0.8242549870873002,  global_disparity:0.0005304068326950073, global_pred_disparity: 0.0018682628870010376,
2023-09-28 23:31:42,713 - utils - INFO - stage3_gradient_single_runtime: 0.006717681884765625
2023-09-28 23:31:42,718 - utils - INFO - 1, epoch: 1640, all client loss: [0.5144078731536865, 0.4562641680240631], all pred client disparities: [0.01024317741394043, 0.0023056119680404663], all client disparities: [0.0032608509063720703, 0.003159552812576294], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([7.1040e-01, 3.8517e-17, 2.8960e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:42,979 - utils - INFO - stage3_gradient_single_runtime: 0.006563425064086914
2023-09-28 23:31:42,984 - utils - INFO - 1, epoch: 1641, all client loss: [0.5143805146217346, 0.4562610685825348], all pred client disparities: [0.01024046540260315, 0.002308383584022522], all client disparities: [0.0032608509063720703, 0.003159552812576294], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([ 7.1077e-01, -7.6859e-17,  2.8923e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:43,250 - utils - INFO - stage3_gradient_single_runtime: 0.01201176643371582
2023-09-28 23:31:43,256 - utils - INFO - 1, epoch: 1642, all client loss: [0.5143532752990723, 0.45625796914100647], all pred client disparities: [0.01023775339126587, 0.002311021089553833], all client disparities: [0.0032608509063720703, 0.003159552812576294], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7111, 0.0000, 0.2889, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:43,594 - utils - INFO - stage3_gradient_single_runtime: 0.0066928863525390625
2023-09-28 23:31:43,598 - utils - INFO - 1, epoch: 1643, all client loss: [0.5143259763717651, 0.4562548100948334], all pred client disparities: [0.010235220193862915, 0.0023135393857955933], all client disparities: [0.0032608509063720703, 0.003159552812576294], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([0.7115, 0.0000, 0.2885, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:43,872 - utils - INFO - stage3_gradient_single_runtime: 0.006697893142700195
2023-09-28 23:31:43,877 - utils - INFO - 1, epoch: 1644, all client loss: [0.5142987370491028, 0.4562515914440155], all pred client disparities: [0.010232716798782349, 0.002316027879714966], all client disparities: [0.0032608509063720703, 0.003159552812576294], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([7.1189e-01, 3.8170e-17, 2.8811e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:44,149 - utils - INFO - stage3_gradient_single_runtime: 0.006452083587646484
2023-09-28 23:31:44,154 - utils - INFO - 1, epoch: 1645, all client loss: [0.5142715573310852, 0.45624834299087524], all pred client disparities: [0.010230273008346558, 0.0023183971643447876], all client disparities: [0.0032608509063720703, 0.003159552812576294], all client accs: [0.7506053447723389, 0.7927398085594177],alphas:tensor([7.1226e-01, 3.8085e-17, 2.8774e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:44,428 - utils - INFO - stage3_gradient_single_runtime: 0.0067403316497802734
2023-09-28 23:31:44,433 - utils - INFO - 1, epoch: 1646, all client loss: [0.5142443180084229, 0.4562450349330902], all pred client disparities: [0.010227948427200317, 0.0023207366466522217], all client disparities: [0.0032608509063720703, 0.002585276961326599], all client accs: [0.7506053447723389, 0.792833149433136],alphas:tensor([7.1262e-01, 3.8001e-17, 2.8738e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:44,700 - utils - INFO - stage3_gradient_single_runtime: 0.006797313690185547
2023-09-28 23:31:44,706 - utils - INFO - 1, epoch: 1647, all client loss: [0.51421719789505, 0.45624175667762756], all pred client disparities: [0.010225653648376465, 0.0023229867219924927], all client disparities: [0.0032608509063720703, 0.002585276961326599], all client accs: [0.7506053447723389, 0.792833149433136],alphas:tensor([0.7130, 0.0000, 0.2870, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:44,970 - utils - INFO - stage3_gradient_single_runtime: 0.0067424774169921875
2023-09-28 23:31:44,975 - utils - INFO - 1, epoch: 1648, all client loss: [0.5141900777816772, 0.45623835921287537], all pred client disparities: [0.010223418474197388, 0.002325102686882019], all client disparities: [0.0032608509063720703, 0.0025226175785064697], all client accs: [0.7506053447723389, 0.792988657951355],alphas:tensor([ 7.1334e-01, -7.5667e-17,  2.8666e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:45,277 - utils - INFO - stage3_gradient_single_runtime: 0.009064435958862305
2023-09-28 23:31:45,283 - utils - INFO - 1, epoch: 1649, all client loss: [0.5141629576683044, 0.4562349319458008], all pred client disparities: [0.010221272706985474, 0.0023271888494491577], all client disparities: [0.0032608509063720703, 0.0025226175785064697], all client accs: [0.7506053447723389, 0.792988657951355],alphas:tensor([0.7137, 0.0000, 0.2863, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:45,559 - utils - INFO - stage3_gradient_single_runtime: 0.006859779357910156
2023-09-28 23:31:45,564 - utils - INFO - 1, epoch: 1650, all client loss: [0.5141359567642212, 0.4562315344810486], all pred client disparities: [0.010219305753707886, 0.002329215407371521], all client disparities: [0.0032608509063720703, 0.0025226175785064697], all client accs: [0.7506053447723389, 0.792988657951355],alphas:tensor([0.7141, 0.0000, 0.2859, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:45,771 - utils - INFO - stage3_gradient_single_runtime: 0.006845951080322266
2023-09-28 23:31:45,777 - utils - INFO - 1, epoch: 1651, all client loss: [0.5141088962554932, 0.4562280774116516], all pred client disparities: [0.01021730899810791, 0.002331152558326721], all client disparities: [0.0032608509063720703, 0.0025226175785064697], all client accs: [0.7506053447723389, 0.7929575443267822],alphas:tensor([0.7144, 0.0000, 0.2856, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:46,051 - utils - INFO - stage3_gradient_single_runtime: 0.006407499313354492
2023-09-28 23:31:46,057 - utils - INFO - 1, epoch: 1652, all client loss: [0.5140818357467651, 0.45622459053993225], all pred client disparities: [0.010215312242507935, 0.002333030104637146], all client disparities: [0.0032608509063720703, 0.0018021464347839355], all client accs: [0.7506053447723389, 0.792988657951355],alphas:tensor([7.1475e-01, 3.7507e-17, 2.8525e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:46,332 - utils - INFO - stage3_gradient_single_runtime: 0.00873708724975586
2023-09-28 23:31:46,337 - utils - INFO - 1, epoch: 1653, all client loss: [0.5140548944473267, 0.4562210738658905], all pred client disparities: [0.010213524103164673, 0.0023348331451416016], all client disparities: [0.0032608509063720703, 0.0018021464347839355], all client accs: [0.7506053447723389, 0.7929575443267822],alphas:tensor([0.7151, 0.0000, 0.2849, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:46,657 - utils - INFO - stage3_gradient_single_runtime: 0.006743192672729492
2023-09-28 23:31:46,664 - utils - INFO - 1, epoch: 1654, all client loss: [0.5140278935432434, 0.456217497587204], all pred client disparities: [0.010211825370788574, 0.002336561679840088], all client disparities: [0.0032608509063720703, 0.0023117661476135254], all client accs: [0.7506053447723389, 0.7934552431106567],alphas:tensor([0.7154, 0.0000, 0.2846, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:46,933 - utils - INFO - stage3_gradient_single_runtime: 0.007324695587158203
2023-09-28 23:31:46,939 - utils - INFO - 1, epoch: 1655, all client loss: [0.5140010118484497, 0.45621392130851746], all pred client disparities: [0.0102100670337677, 0.002338200807571411], all client disparities: [0.0032608509063720703, 0.0023117661476135254], all client accs: [0.7506053447723389, 0.7934552431106567],alphas:tensor([7.1579e-01, 3.7267e-17, 2.8421e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:47,199 - utils - INFO - stage3_gradient_single_runtime: 0.006551265716552734
2023-09-28 23:31:47,204 - utils - INFO - 1, epoch: 1656, all client loss: [0.5139740705490112, 0.45621034502983093], all pred client disparities: [0.010208457708358765, 0.0023398101329803467], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7934552431106567],alphas:tensor([ 7.1613e-01, -7.4378e-17,  2.8387e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:47,453 - utils - INFO - stage3_gradient_single_runtime: 0.006354331970214844
2023-09-28 23:31:47,456 - utils - INFO - 1, epoch: 1657, all client loss: [0.5139471888542175, 0.45620667934417725], all pred client disparities: [0.010206878185272217, 0.002341344952583313], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793424129486084],alphas:tensor([7.1646e-01, 3.7111e-17, 2.8354e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:47,707 - utils - INFO - stage3_gradient_single_runtime: 0.00626683235168457
2023-09-28 23:31:47,711 - utils - INFO - 1, epoch: 1658, all client loss: [0.5139203071594238, 0.45620304346084595], all pred client disparities: [0.010205388069152832, 0.00234280526638031], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793424129486084],alphas:tensor([0.7168, 0.0000, 0.2832, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:47,960 - utils - INFO - stage3_gradient_single_runtime: 0.006289482116699219
2023-09-28 23:31:47,963 - utils - INFO - 1, epoch: 1659, all client loss: [0.5138934850692749, 0.45619940757751465], all pred client disparities: [0.010203957557678223, 0.0023442208766937256], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932997345924377],alphas:tensor([0.7171, 0.0000, 0.2829, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:48,111 - utils - INFO - valid: True, epoch: 1659, loss: [0.5768147706985474, 0.45537373423576355], accuracy: [0.7348066568374634, 0.7947826385498047], mean_accuracy:0.764794647693634,variance_accuracy:0.029987990856170654, disparity: [0.004545450210571289, 0.00029064714908599854], mean_disparity:0.002418048679828644,variance_disparity:0.0021274015307426453, pred_disparity: [0.004440635442733765, 0.0013122409582138062]
2023-09-28 23:31:48,196 - utils - INFO - global_valid: True, epoch: 1659,  global_loss: 0.4567238390445709, global_accuracy: 0.8247725091955025,  global_disparity:0.0030528604984283447, global_pred_disparity: 0.0019692182540893555,
2023-09-28 23:31:48,470 - utils - INFO - stage3_gradient_single_runtime: 0.006315946578979492
2023-09-28 23:31:48,474 - utils - INFO - 1, epoch: 1660, all client loss: [0.5138667225837708, 0.4561956822872162], all pred client disparities: [0.010202616453170776, 0.002345547080039978], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793268620967865],alphas:tensor([0.6678, 0.3322, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:48,734 - utils - INFO - stage3_gradient_single_runtime: 0.0063402652740478516
2023-09-28 23:31:48,739 - utils - INFO - 1, epoch: 1661, all client loss: [0.5138401389122009, 0.4559970200061798], all pred client disparities: [0.01026219129562378, 0.002286389470100403], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793424129486084],alphas:tensor([7.1554e-01, 3.7326e-17, 2.8446e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:48,999 - utils - INFO - stage3_gradient_single_runtime: 0.00640869140625
2023-09-28 23:31:49,005 - utils - INFO - 1, epoch: 1662, all client loss: [0.5138133764266968, 0.4559946656227112], all pred client disparities: [0.010260134935379028, 0.002288445830345154], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793393075466156],alphas:tensor([7.1590e-01, 3.7244e-17, 2.8410e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:49,267 - utils - INFO - stage3_gradient_single_runtime: 0.006503582000732422
2023-09-28 23:31:49,273 - utils - INFO - 1, epoch: 1663, all client loss: [0.5137866735458374, 0.4559922516345978], all pred client disparities: [0.010258078575134277, 0.0022904425859451294], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793268620967865],alphas:tensor([0.7163, 0.0000, 0.2837, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:49,531 - utils - INFO - stage3_gradient_single_runtime: 0.006432533264160156
2023-09-28 23:31:49,536 - utils - INFO - 1, epoch: 1664, all client loss: [0.513759970664978, 0.455989807844162], all pred client disparities: [0.010256201028823853, 0.0022923648357391357], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793268620967865],alphas:tensor([0.7166, 0.0000, 0.2834, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:49,791 - utils - INFO - stage3_gradient_single_runtime: 0.00641179084777832
2023-09-28 23:31:49,796 - utils - INFO - 1, epoch: 1665, all client loss: [0.5137333273887634, 0.4559873640537262], all pred client disparities: [0.010254234075546265, 0.002294197678565979], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932997345924377],alphas:tensor([0.7170, 0.0000, 0.2830, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:50,048 - utils - INFO - stage3_gradient_single_runtime: 0.006409168243408203
2023-09-28 23:31:50,053 - utils - INFO - 1, epoch: 1666, all client loss: [0.5137066841125488, 0.45598486065864563], all pred client disparities: [0.010252505540847778, 0.002295941114425659], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932997345924377],alphas:tensor([7.1732e-01, 3.6920e-17, 2.8268e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:50,304 - utils - INFO - stage3_gradient_single_runtime: 0.006422758102416992
2023-09-28 23:31:50,309 - utils - INFO - 1, epoch: 1667, all client loss: [0.5136800408363342, 0.4559823274612427], all pred client disparities: [0.010250717401504517, 0.0022976547479629517], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932997345924377],alphas:tensor([0.7177, 0.0000, 0.2823, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:50,562 - utils - INFO - stage3_gradient_single_runtime: 0.006501913070678711
2023-09-28 23:31:50,567 - utils - INFO - 1, epoch: 1668, all client loss: [0.5136534571647644, 0.45597976446151733], all pred client disparities: [0.010249108076095581, 0.002299293875694275], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932997345924377],alphas:tensor([0.7180, 0.0000, 0.2820, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:50,823 - utils - INFO - stage3_gradient_single_runtime: 0.006513357162475586
2023-09-28 23:31:50,828 - utils - INFO - 1, epoch: 1669, all client loss: [0.5136269330978394, 0.4559771716594696], all pred client disparities: [0.010247468948364258, 0.0023008137941360474], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932997345924377],alphas:tensor([0.7184, 0.0000, 0.2816, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:51,082 - utils - INFO - stage3_gradient_single_runtime: 0.0064487457275390625
2023-09-28 23:31:51,087 - utils - INFO - 1, epoch: 1670, all client loss: [0.5136003494262695, 0.4559745490550995], all pred client disparities: [0.010245949029922485, 0.0023023486137390137], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932997345924377],alphas:tensor([0.7187, 0.0000, 0.2813, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:51,346 - utils - INFO - stage3_gradient_single_runtime: 0.0064966678619384766
2023-09-28 23:31:51,351 - utils - INFO - 1, epoch: 1671, all client loss: [0.513573944568634, 0.455971896648407], all pred client disparities: [0.010244548320770264, 0.0023037344217300415], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793268620967865],alphas:tensor([0.7190, 0.0000, 0.2810, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:51,613 - utils - INFO - stage3_gradient_single_runtime: 0.00654292106628418
2023-09-28 23:31:51,616 - utils - INFO - 1, epoch: 1672, all client loss: [0.5135474801063538, 0.4559692144393921], all pred client disparities: [0.010243207216262817, 0.0023050904273986816], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793268620967865],alphas:tensor([0.7194, 0.0000, 0.2806, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:51,868 - utils - INFO - stage3_gradient_single_runtime: 0.006296634674072266
2023-09-28 23:31:51,873 - utils - INFO - 1, epoch: 1673, all client loss: [0.5135210156440735, 0.4559665322303772], all pred client disparities: [0.010241776704788208, 0.0023063868284225464], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793268620967865],alphas:tensor([0.7197, 0.0000, 0.2803, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:52,126 - utils - INFO - stage3_gradient_single_runtime: 0.006352424621582031
2023-09-28 23:31:52,131 - utils - INFO - 1, epoch: 1674, all client loss: [0.513494610786438, 0.45596379041671753], all pred client disparities: [0.010240554809570312, 0.0023076236248016357], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.793268620967865],alphas:tensor([0.7200, 0.0000, 0.2800, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:52,382 - utils - INFO - stage3_gradient_single_runtime: 0.006323814392089844
2023-09-28 23:31:52,385 - utils - INFO - 1, epoch: 1675, all client loss: [0.5134681463241577, 0.45596110820770264], all pred client disparities: [0.010239362716674805, 0.002308771014213562], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932063937187195],alphas:tensor([7.2035e-01, 3.6226e-17, 2.7965e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:52,637 - utils - INFO - stage3_gradient_single_runtime: 0.006308317184448242
2023-09-28 23:31:52,641 - utils - INFO - 1, epoch: 1676, all client loss: [0.5134418606758118, 0.4559583365917206], all pred client disparities: [0.010238230228424072, 0.002309858798980713], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932063937187195],alphas:tensor([0.7207, 0.0000, 0.2793, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:52,897 - utils - INFO - stage3_gradient_single_runtime: 0.006394147872924805
2023-09-28 23:31:52,900 - utils - INFO - 1, epoch: 1677, all client loss: [0.513415515422821, 0.4559555649757385], all pred client disparities: [0.010237187147140503, 0.0023108869791030884], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932063937187195],alphas:tensor([ 7.2100e-01, -7.2155e-17,  2.7900e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:53,152 - utils - INFO - stage3_gradient_single_runtime: 0.00630640983581543
2023-09-28 23:31:53,157 - utils - INFO - 1, epoch: 1678, all client loss: [0.5133891701698303, 0.4559527635574341], all pred client disparities: [0.010236114263534546, 0.002311885356903076], all client disparities: [0.0032608509063720703, 0.0025310367345809937], all client accs: [0.7506053447723389, 0.7932063937187195],alphas:tensor([7.2132e-01, 3.6004e-17, 2.7868e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:53,432 - utils - INFO - stage3_gradient_single_runtime: 0.010321617126464844
2023-09-28 23:31:53,438 - utils - INFO - 1, epoch: 1679, all client loss: [0.5133628845214844, 0.45594993233680725], all pred client disparities: [0.01023527979850769, 0.002312779426574707], all client disparities: [0.0032608509063720703, 0.002457946538925171], all client accs: [0.7506053447723389, 0.7932375073432922],alphas:tensor([7.2164e-01, 3.5932e-17, 2.7836e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:53,523 - utils - INFO - valid: True, epoch: 1679, loss: [0.5766711831092834, 0.4551104009151459], accuracy: [0.7348066568374634, 0.7947826385498047], mean_accuracy:0.764794647693634,variance_accuracy:0.029987990856170654, disparity: [0.004545450210571289, 0.0005865097045898438], mean_disparity:0.0025659799575805664,variance_disparity:0.0019794702529907227, pred_disparity: [0.004447340965270996, 0.0011113882064819336]
2023-09-28 23:31:53,715 - utils - INFO - global_valid: True, epoch: 1679,  global_loss: 0.4564618468284607, global_accuracy: 0.8251513486353689,  global_disparity:0.002766326069831848, global_pred_disparity: 0.0021440833806991577,
2023-09-28 23:31:53,954 - utils - INFO - stage3_gradient_single_runtime: 0.0063953399658203125
2023-09-28 23:31:53,959 - utils - INFO - 1, epoch: 1680, all client loss: [0.5133366584777832, 0.4559471607208252], all pred client disparities: [0.010234296321868896, 0.002313658595085144], all client disparities: [0.0032608509063720703, 0.002457946538925171], all client accs: [0.7506053447723389, 0.7932375073432922],alphas:tensor([7.2195e-01, 3.5859e-17, 2.7805e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:54,200 - utils - INFO - stage3_gradient_single_runtime: 0.006393909454345703
2023-09-28 23:31:54,205 - utils - INFO - 1, epoch: 1681, all client loss: [0.513310432434082, 0.455944299697876], all pred client disparities: [0.010233521461486816, 0.002314463257789612], all client disparities: [0.0032608509063720703, 0.008150234818458557], all client accs: [0.7506053447723389, 0.7976856827735901],alphas:tensor([7.2227e-01, 3.5788e-17, 2.7773e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:54,445 - utils - INFO - stage3_gradient_single_runtime: 0.006362438201904297
2023-09-28 23:31:54,450 - utils - INFO - 1, epoch: 1682, all client loss: [0.5132842063903809, 0.45594143867492676], all pred client disparities: [0.010232716798782349, 0.002315208315849304], all client disparities: [0.0032608509063720703, 0.008150234818458557], all client accs: [0.7506053447723389, 0.7976856827735901],alphas:tensor([0.7226, 0.0000, 0.2774, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:54,692 - utils - INFO - stage3_gradient_single_runtime: 0.006447792053222656
2023-09-28 23:31:54,697 - utils - INFO - 1, epoch: 1683, all client loss: [0.5132580399513245, 0.45593857765197754], all pred client disparities: [0.010232031345367432, 0.0023158788681030273], all client disparities: [0.0032608509063720703, 0.008150234818458557], all client accs: [0.7506053447723389, 0.7976856827735901],alphas:tensor([0.7229, 0.0000, 0.2771, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:54,941 - utils - INFO - stage3_gradient_single_runtime: 0.006350040435791016
2023-09-28 23:31:54,946 - utils - INFO - 1, epoch: 1684, all client loss: [0.5132319331169128, 0.4559357166290283], all pred client disparities: [0.010231316089630127, 0.002316504716873169], all client disparities: [0.0032608509063720703, 0.008150234818458557], all client accs: [0.7506053447723389, 0.7976856827735901],alphas:tensor([7.2320e-01, 3.5576e-17, 2.7680e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:55,190 - utils - INFO - stage3_gradient_single_runtime: 0.006394863128662109
2023-09-28 23:31:55,195 - utils - INFO - 1, epoch: 1685, all client loss: [0.5132057666778564, 0.4559328258037567], all pred client disparities: [0.010230720043182373, 0.002317085862159729], all client disparities: [0.0032608509063720703, 0.008150234818458557], all client accs: [0.7506053447723389, 0.7975612878799438],alphas:tensor([0.7235, 0.0000, 0.2765, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:55,435 - utils - INFO - stage3_gradient_single_runtime: 0.006414175033569336
2023-09-28 23:31:55,440 - utils - INFO - 1, epoch: 1686, all client loss: [0.5131797194480896, 0.4559299647808075], all pred client disparities: [0.010230213403701782, 0.0023176074028015137], all client disparities: [0.0032608509063720703, 0.008150234818458557], all client accs: [0.7506053447723389, 0.7975612878799438],alphas:tensor([0.7238, 0.0000, 0.2762, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:55,681 - utils - INFO - stage3_gradient_single_runtime: 0.006417512893676758
2023-09-28 23:31:55,686 - utils - INFO - 1, epoch: 1687, all client loss: [0.513153612613678, 0.4559270441532135], all pred client disparities: [0.010229706764221191, 0.002318069338798523], all client disparities: [0.0032608509063720703, 0.0075028687715530396], all client accs: [0.7506053447723389, 0.7975301742553711],alphas:tensor([ 7.2410e-01, -7.0737e-17,  2.7590e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:55,926 - utils - INFO - stage3_gradient_single_runtime: 0.00637507438659668
2023-09-28 23:31:55,931 - utils - INFO - 1, epoch: 1688, all client loss: [0.5131276249885559, 0.4559241235256195], all pred client disparities: [0.010229289531707764, 0.0023185163736343384], all client disparities: [0.0032608509063720703, 0.006782412528991699], all client accs: [0.7506053447723389, 0.7975612878799438],alphas:tensor([0.7244, 0.0000, 0.2756, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:56,173 - utils - INFO - stage3_gradient_single_runtime: 0.0065610408782958984
2023-09-28 23:31:56,178 - utils - INFO - 1, epoch: 1689, all client loss: [0.5131015777587891, 0.4559212028980255], all pred client disparities: [0.010228872299194336, 0.002318829298019409], all client disparities: [0.0032608509063720703, 0.004621028900146484], all client accs: [0.7506053447723389, 0.7976545691490173],alphas:tensor([0.7247, 0.0000, 0.2753, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:56,419 - utils - INFO - stage3_gradient_single_runtime: 0.0064334869384765625
2023-09-28 23:31:56,425 - utils - INFO - 1, epoch: 1690, all client loss: [0.513075590133667, 0.4559182822704315], all pred client disparities: [0.010228604078292847, 0.0023191124200820923], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7977167963981628],alphas:tensor([0.7250, 0.0000, 0.2750, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:56,667 - utils - INFO - stage3_gradient_single_runtime: 0.006510496139526367
2023-09-28 23:31:56,672 - utils - INFO - 1, epoch: 1691, all client loss: [0.5130496621131897, 0.4559153616428375], all pred client disparities: [0.010228246450424194, 0.0023194104433059692], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7976856827735901],alphas:tensor([0.7253, 0.0000, 0.2747, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:56,914 - utils - INFO - stage3_gradient_single_runtime: 0.006420135498046875
2023-09-28 23:31:56,919 - utils - INFO - 1, epoch: 1692, all client loss: [0.5130237340927124, 0.45591244101524353], all pred client disparities: [0.01022803783416748, 0.0023196041584014893], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7976856827735901],alphas:tensor([0.7256, 0.0000, 0.2744, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:57,160 - utils - INFO - stage3_gradient_single_runtime: 0.006439685821533203
2023-09-28 23:31:57,165 - utils - INFO - 1, epoch: 1693, all client loss: [0.5129978060722351, 0.45590949058532715], all pred client disparities: [0.010227888822555542, 0.0023197680711746216], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7976235151290894],alphas:tensor([0.7259, 0.0000, 0.2741, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:57,406 - utils - INFO - stage3_gradient_single_runtime: 0.006422758102416992
2023-09-28 23:31:57,411 - utils - INFO - 1, epoch: 1694, all client loss: [0.5129719376564026, 0.45590656995773315], all pred client disparities: [0.010227739810943604, 0.0023198723793029785], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7976235151290894],alphas:tensor([0.7262, 0.0000, 0.2738, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:57,645 - utils - INFO - stage3_gradient_single_runtime: 0.006490468978881836
2023-09-28 23:31:57,647 - utils - INFO - 1, epoch: 1695, all client loss: [0.5129460692405701, 0.4559036195278168], all pred client disparities: [0.01022765040397644, 0.0023199468851089478], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7976235151290894],alphas:tensor([7.2643e-01, 3.4837e-17, 2.7357e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:57,887 - utils - INFO - stage3_gradient_single_runtime: 0.006468772888183594
2023-09-28 23:31:57,892 - utils - INFO - 1, epoch: 1696, all client loss: [0.5129202604293823, 0.4559006690979004], all pred client disparities: [0.010227590799331665, 0.002319931983947754], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7976235151290894],alphas:tensor([7.2672e-01, 3.4772e-17, 2.7328e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:58,134 - utils - INFO - stage3_gradient_single_runtime: 0.00651240348815918
2023-09-28 23:31:58,139 - utils - INFO - 1, epoch: 1697, all client loss: [0.5128944516181946, 0.4558977484703064], all pred client disparities: [0.010227620601654053, 0.00231991708278656], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7976235151290894],alphas:tensor([ 7.2700e-01, -6.9417e-17,  2.7300e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:58,378 - utils - INFO - stage3_gradient_single_runtime: 0.006478071212768555
2023-09-28 23:31:58,383 - utils - INFO - 1, epoch: 1698, all client loss: [0.5128687024116516, 0.4558948278427124], all pred client disparities: [0.010227710008621216, 0.002319827675819397], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7481840252876282, 0.7976235151290894],alphas:tensor([0.6879, 0.3121, 0.0000, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:58,622 - utils - INFO - stage3_gradient_single_runtime: 0.006523609161376953
2023-09-28 23:31:58,627 - utils - INFO - 1, epoch: 1699, all client loss: [0.5128431916236877, 0.45570340752601624], all pred client disparities: [0.010284066200256348, 0.0022639185190200806], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7506053447723389, 0.7977790236473083],alphas:tensor([7.2529e-01, 3.5090e-17, 2.7471e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:31:58,709 - utils - INFO - valid: True, epoch: 1699, loss: [0.5765106678009033, 0.45485028624534607], accuracy: [0.7348066568374634, 0.8004968762397766], mean_accuracy:0.76765176653862,variance_accuracy:0.032845109701156616, disparity: [0.004545450210571289, 0.004401862621307373], mean_disparity:0.004473656415939331,variance_disparity:7.179379463195801e-05, pred_disparity: [0.004462897777557373, 0.0008697062730789185]
2023-09-28 23:31:58,839 - utils - INFO - global_valid: True, epoch: 1699,  global_loss: 0.45620280504226685, global_accuracy: 0.8254376005357866,  global_disparity:0.0010688602924346924, global_pred_disparity: 0.00235883891582489,
2023-09-28 23:31:59,081 - utils - INFO - stage3_gradient_single_runtime: 0.006496429443359375
2023-09-28 23:31:59,085 - utils - INFO - 1, epoch: 1700, all client loss: [0.5128175020217896, 0.4557018280029297], all pred client disparities: [0.010283350944519043, 0.0022645145654678345], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7506053447723389, 0.7977790236473083],alphas:tensor([0.7256, 0.0000, 0.2744, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:59,325 - utils - INFO - stage3_gradient_single_runtime: 0.006454944610595703
2023-09-28 23:31:59,331 - utils - INFO - 1, epoch: 1701, all client loss: [0.5127918124198914, 0.45570024847984314], all pred client disparities: [0.010282844305038452, 0.0022650808095932007], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7506053447723389, 0.7976235151290894],alphas:tensor([0.7259, 0.0000, 0.2741, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:59,524 - utils - INFO - stage3_gradient_single_runtime: 0.006360769271850586
2023-09-28 23:31:59,529 - utils - INFO - 1, epoch: 1702, all client loss: [0.5127661824226379, 0.4556986093521118], all pred client disparities: [0.010282248258590698, 0.0022655874490737915], all client disparities: [0.0032608509063720703, 0.00476720929145813], all client accs: [0.7506053447723389, 0.7976235151290894],alphas:tensor([0.7262, 0.0000, 0.2738, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:31:59,766 - utils - INFO - stage3_gradient_single_runtime: 0.0062618255615234375
2023-09-28 23:31:59,771 - utils - INFO - 1, epoch: 1703, all client loss: [0.5127405524253845, 0.4556969702243805], all pred client disparities: [0.010281771421432495, 0.002266019582748413], all client disparities: [0.0032608509063720703, 0.004840284585952759], all client accs: [0.7506053447723389, 0.7976545691490173],alphas:tensor([0.7265, 0.0000, 0.2735, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:00,010 - utils - INFO - stage3_gradient_single_runtime: 0.007536172866821289
2023-09-28 23:32:00,015 - utils - INFO - 1, epoch: 1704, all client loss: [0.5127149820327759, 0.4556953012943268], all pred client disparities: [0.010281354188919067, 0.002266421914100647], all client disparities: [0.0032608509063720703, 0.004840284585952759], all client accs: [0.7506053447723389, 0.7976545691490173],alphas:tensor([7.2681e-01, 3.4747e-17, 2.7319e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:00,264 - utils - INFO - stage3_gradient_single_runtime: 0.007268667221069336
2023-09-28 23:32:00,270 - utils - INFO - 1, epoch: 1705, all client loss: [0.512689471244812, 0.45569363236427307], all pred client disparities: [0.010281085968017578, 0.0022667497396469116], all client disparities: [0.0032608509063720703, 0.005560740828514099], all client accs: [0.7506053447723389, 0.7976235151290894],alphas:tensor([7.2711e-01, 3.4679e-17, 2.7289e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:00,520 - utils - INFO - stage3_gradient_single_runtime: 0.0071833133697509766
2023-09-28 23:32:00,524 - utils - INFO - 1, epoch: 1706, all client loss: [0.5126639604568481, 0.455691933631897], all pred client disparities: [0.010280758142471313, 0.0022670477628707886], all client disparities: [0.0032608509063720703, 0.004840284585952759], all client accs: [0.7506053447723389, 0.7976545691490173],alphas:tensor([7.2740e-01, 3.4613e-17, 2.7260e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:00,784 - utils - INFO - stage3_gradient_single_runtime: 0.00690913200378418
2023-09-28 23:32:00,789 - utils - INFO - 1, epoch: 1707, all client loss: [0.5126384496688843, 0.4556902348995209], all pred client disparities: [0.0102805495262146, 0.0022672563791275024], all client disparities: [0.0032608509063720703, 0.005560740828514099], all client accs: [0.7506053447723389, 0.7975612878799438],alphas:tensor([0.7277, 0.0000, 0.2723, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:01,032 - utils - INFO - stage3_gradient_single_runtime: 0.006999015808105469
2023-09-28 23:32:01,038 - utils - INFO - 1, epoch: 1708, all client loss: [0.5126129984855652, 0.4556885063648224], all pred client disparities: [0.010280311107635498, 0.0022674351930618286], all client disparities: [0.0032608509063720703, 0.005560740828514099], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([7.2798e-01, 3.4481e-17, 2.7202e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:01,283 - utils - INFO - stage3_gradient_single_runtime: 0.006299734115600586
2023-09-28 23:32:01,288 - utils - INFO - 1, epoch: 1709, all client loss: [0.5125875473022461, 0.4556867778301239], all pred client disparities: [0.01028016209602356, 0.0022675544023513794], all client disparities: [0.0032608509063720703, 0.005560740828514099], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([0.7283, 0.0000, 0.2717, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:01,528 - utils - INFO - stage3_gradient_single_runtime: 0.006274700164794922
2023-09-28 23:32:01,530 - utils - INFO - 1, epoch: 1710, all client loss: [0.5125621557235718, 0.45568498969078064], all pred client disparities: [0.010280013084411621, 0.002267599105834961], all client disparities: [0.0032608509063720703, 0.005560740828514099], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([0.7286, 0.0000, 0.2714, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:01,772 - utils - INFO - stage3_gradient_single_runtime: 0.006533145904541016
2023-09-28 23:32:01,774 - utils - INFO - 1, epoch: 1711, all client loss: [0.5125367641448975, 0.45568329095840454], all pred client disparities: [0.010280102491378784, 0.0022676289081573486], all client disparities: [0.0032608509063720703, 0.005560740828514099], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([0.7288, 0.0000, 0.2712, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:02,020 - utils - INFO - stage3_gradient_single_runtime: 0.006261348724365234
2023-09-28 23:32:02,023 - utils - INFO - 1, epoch: 1712, all client loss: [0.5125114321708679, 0.45568153262138367], all pred client disparities: [0.010280042886734009, 0.002267599105834961], all client disparities: [0.0032608509063720703, 0.005560740828514099], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([0.7291, 0.0000, 0.2709, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:02,274 - utils - INFO - stage3_gradient_single_runtime: 0.006283760070800781
2023-09-28 23:32:02,278 - utils - INFO - 1, epoch: 1713, all client loss: [0.5124861598014832, 0.4556798040866852], all pred client disparities: [0.01028016209602356, 0.002267509698867798], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7975612878799438],alphas:tensor([0.7294, 0.0000, 0.2706, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:02,528 - utils - INFO - stage3_gradient_single_runtime: 0.006239891052246094
2023-09-28 23:32:02,533 - utils - INFO - 1, epoch: 1714, all client loss: [0.5124608278274536, 0.4556780159473419], all pred client disparities: [0.010280191898345947, 0.0022673755884170532], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([0.7297, 0.0000, 0.2703, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:02,770 - utils - INFO - stage3_gradient_single_runtime: 0.006899595260620117
2023-09-28 23:32:02,776 - utils - INFO - 1, epoch: 1715, all client loss: [0.5124355554580688, 0.45567625761032104], all pred client disparities: [0.010280400514602661, 0.002267181873321533], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([7.2995e-01, 3.4037e-17, 2.7005e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:03,025 - utils - INFO - stage3_gradient_single_runtime: 0.007302761077880859
2023-09-28 23:32:03,031 - utils - INFO - 1, epoch: 1716, all client loss: [0.5124103426933289, 0.45567449927330017], all pred client disparities: [0.010280609130859375, 0.002266988158226013], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([0.7302, 0.0000, 0.2698, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:03,281 - utils - INFO - stage3_gradient_single_runtime: 0.007222652435302734
2023-09-28 23:32:03,286 - utils - INFO - 1, epoch: 1717, all client loss: [0.5123851895332336, 0.4556727111339569], all pred client disparities: [0.010280847549438477, 0.0022666752338409424], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7974368333816528],alphas:tensor([7.3050e-01, 3.3914e-17, 2.6950e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:03,536 - utils - INFO - stage3_gradient_single_runtime: 0.007564067840576172
2023-09-28 23:32:03,542 - utils - INFO - 1, epoch: 1718, all client loss: [0.5123599767684937, 0.45567092299461365], all pred client disparities: [0.010281145572662354, 0.0022663772106170654], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7974368333816528],alphas:tensor([7.3077e-01, 3.3854e-17, 2.6923e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:03,788 - utils - INFO - stage3_gradient_single_runtime: 0.007170915603637695
2023-09-28 23:32:03,793 - utils - INFO - 1, epoch: 1719, all client loss: [0.5123348236083984, 0.4556691348552704], all pred client disparities: [0.010281503200531006, 0.0022660046815872192], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7974368333816528],alphas:tensor([0.7310, 0.0000, 0.2690, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:03,925 - utils - INFO - valid: True, epoch: 1719, loss: [0.5764390230178833, 0.45480746030807495], accuracy: [0.7348066568374634, 0.8004347681999207], mean_accuracy:0.767620712518692,variance_accuracy:0.03281405568122864, disparity: [0.004545450210571289, 0.003231436014175415], mean_disparity:0.003888443112373352,variance_disparity:0.000657007098197937, pred_disparity: [0.004439353942871094, 0.000613972544670105]
2023-09-28 23:32:04,016 - utils - INFO - global_valid: True, epoch: 1719,  global_loss: 0.45615968108177185, global_accuracy: 0.8257430055947714,  global_disparity:4.9099329771706834e-05, global_pred_disparity: 0.002582177519798279,
2023-09-28 23:32:04,255 - utils - INFO - stage3_gradient_single_runtime: 0.0069158077239990234
2023-09-28 23:32:04,261 - utils - INFO - 1, epoch: 1720, all client loss: [0.512309730052948, 0.4556673765182495], all pred client disparities: [0.010281890630722046, 0.002265557646751404], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7974368333816528],alphas:tensor([ 7.3130e-01, -6.7468e-17,  2.6870e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:04,503 - utils - INFO - stage3_gradient_single_runtime: 0.00644373893737793
2023-09-28 23:32:04,509 - utils - INFO - 1, epoch: 1721, all client loss: [0.5122846961021423, 0.45566561818122864], all pred client disparities: [0.010282337665557861, 0.0022651106119155884], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7974368333816528],alphas:tensor([0.7316, 0.0000, 0.2684, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:04,751 - utils - INFO - stage3_gradient_single_runtime: 0.006284475326538086
2023-09-28 23:32:04,754 - utils - INFO - 1, epoch: 1722, all client loss: [0.5122596025466919, 0.455663800239563], all pred client disparities: [0.010282844305038452, 0.0022645890712738037], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7974368333816528],alphas:tensor([0.7318, 0.0000, 0.2682, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:04,996 - utils - INFO - stage3_gradient_single_runtime: 0.0063550472259521484
2023-09-28 23:32:05,000 - utils - INFO - 1, epoch: 1723, all client loss: [0.5122345685958862, 0.4556620121002197], all pred client disparities: [0.01028338074684143, 0.002264067530632019], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7974368333816528],alphas:tensor([0.7321, 0.0000, 0.2679, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:05,244 - utils - INFO - stage3_gradient_single_runtime: 0.006259918212890625
2023-09-28 23:32:05,248 - utils - INFO - 1, epoch: 1724, all client loss: [0.5122095942497253, 0.45566022396087646], all pred client disparities: [0.010283976793289185, 0.002263486385345459], all client disparities: [0.0032608509063720703, 0.005633831024169922], all client accs: [0.7481840252876282, 0.7974368333816528],alphas:tensor([0.7323, 0.0000, 0.2677, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:05,496 - utils - INFO - stage3_gradient_single_runtime: 0.0063359737396240234
2023-09-28 23:32:05,500 - utils - INFO - 1, epoch: 1725, all client loss: [0.5121845602989197, 0.4556584358215332], all pred client disparities: [0.010284572839736938, 0.002262815833091736], all client disparities: [0.0032608509063720703, 0.006521329283714294], all client accs: [0.7481840252876282, 0.7977479100227356],alphas:tensor([0.7326, 0.0000, 0.2674, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:05,742 - utils - INFO - stage3_gradient_single_runtime: 0.006370067596435547
2023-09-28 23:32:05,746 - utils - INFO - 1, epoch: 1726, all client loss: [0.5121597051620483, 0.45565664768218994], all pred client disparities: [0.01028519868850708, 0.0022621601819992065], all client disparities: [0.0032608509063720703, 0.006521329283714294], all client accs: [0.7481840252876282, 0.7977479100227356],alphas:tensor([7.3284e-01, 3.3385e-17, 2.6716e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:05,987 - utils - INFO - stage3_gradient_single_runtime: 0.006236553192138672
2023-09-28 23:32:05,992 - utils - INFO - 1, epoch: 1727, all client loss: [0.5121347904205322, 0.45565488934516907], all pred client disparities: [0.010285943746566772, 0.002261444926261902], all client disparities: [0.0032608509063720703, 0.006521329283714294], all client accs: [0.7481840252876282, 0.7977479100227356],alphas:tensor([0.7331, 0.0000, 0.2669, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:06,233 - utils - INFO - stage3_gradient_single_runtime: 0.006224393844604492
2023-09-28 23:32:06,238 - utils - INFO - 1, epoch: 1728, all client loss: [0.5121098756790161, 0.4556531012058258], all pred client disparities: [0.01028662919998169, 0.0022606700658798218], all client disparities: [0.0032608509063720703, 0.012253537774085999], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([0.7333, 0.0000, 0.2667, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:06,480 - utils - INFO - stage3_gradient_single_runtime: 0.006450653076171875
2023-09-28 23:32:06,486 - utils - INFO - 1, epoch: 1729, all client loss: [0.5120850205421448, 0.45565134286880493], all pred client disparities: [0.010287463665008545, 0.0022598952054977417], all client disparities: [0.0032608509063720703, 0.012253537774085999], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([0.7336, 0.0000, 0.2664, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:06,725 - utils - INFO - stage3_gradient_single_runtime: 0.007488250732421875
2023-09-28 23:32:06,731 - utils - INFO - 1, epoch: 1730, all client loss: [0.5120602250099182, 0.45564958453178406], all pred client disparities: [0.010288268327713013, 0.0022590458393096924], all client disparities: [0.0032608509063720703, 0.012253537774085999], all client accs: [0.7481840252876282, 0.7983078360557556],alphas:tensor([0.7338, 0.0000, 0.2662, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:06,990 - utils - INFO - stage3_gradient_single_runtime: 0.006539106369018555
2023-09-28 23:32:06,996 - utils - INFO - 1, epoch: 1731, all client loss: [0.5120354890823364, 0.4556478261947632], all pred client disparities: [0.010289162397384644, 0.0022581666707992554], all client disparities: [0.0032608509063720703, 0.012253537774085999], all client accs: [0.7481840252876282, 0.7983078360557556],alphas:tensor([0.7341, 0.0000, 0.2659, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:07,247 - utils - INFO - stage3_gradient_single_runtime: 0.0062847137451171875
2023-09-28 23:32:07,252 - utils - INFO - 1, epoch: 1732, all client loss: [0.5120106935501099, 0.4556460380554199], all pred client disparities: [0.010290086269378662, 0.002257242798805237], all client disparities: [0.0032608509063720703, 0.012253537774085999], all client accs: [0.7481840252876282, 0.7983078360557556],alphas:tensor([7.3431e-01, 3.3052e-17, 2.6569e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:07,490 - utils - INFO - stage3_gradient_single_runtime: 0.006256580352783203
2023-09-28 23:32:07,496 - utils - INFO - 1, epoch: 1733, all client loss: [0.5119859576225281, 0.45564430952072144], all pred client disparities: [0.010290980339050293, 0.0022562891244888306], all client disparities: [0.0032608509063720703, 0.012253537774085999], all client accs: [0.7481840252876282, 0.7983078360557556],alphas:tensor([0.7345, 0.0000, 0.2655, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:07,732 - utils - INFO - stage3_gradient_single_runtime: 0.007212162017822266
2023-09-28 23:32:07,738 - utils - INFO - 1, epoch: 1734, all client loss: [0.5119612216949463, 0.45564258098602295], all pred client disparities: [0.0102919340133667, 0.0022552907466888428], all client disparities: [0.0032608509063720703, 0.012253537774085999], all client accs: [0.7481840252876282, 0.7982767224311829],alphas:tensor([ 7.3478e-01, -6.5889e-17,  2.6522e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:07,984 - utils - INFO - stage3_gradient_single_runtime: 0.007179975509643555
2023-09-28 23:32:07,990 - utils - INFO - 1, epoch: 1735, all client loss: [0.511936604976654, 0.4556408226490021], all pred client disparities: [0.01029294729232788, 0.0022542327642440796], all client disparities: [0.0032608509063720703, 0.010812610387802124], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([0.7350, 0.0000, 0.2650, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:08,241 - utils - INFO - stage3_gradient_single_runtime: 0.006288766860961914
2023-09-28 23:32:08,246 - utils - INFO - 1, epoch: 1736, all client loss: [0.511911928653717, 0.455639123916626], all pred client disparities: [0.010294079780578613, 0.002253204584121704], all client disparities: [0.0032608509063720703, 0.010812610387802124], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([0.7352, 0.0000, 0.2648, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:08,482 - utils - INFO - stage3_gradient_single_runtime: 0.006548166275024414
2023-09-28 23:32:08,486 - utils - INFO - 1, epoch: 1737, all client loss: [0.5118873119354248, 0.4556373953819275], all pred client disparities: [0.01029515266418457, 0.0022520869970321655], all client disparities: [0.0032608509063720703, 0.010812610387802124], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([7.3548e-01, 3.2786e-17, 2.6452e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:08,732 - utils - INFO - stage3_gradient_single_runtime: 0.006261110305786133
2023-09-28 23:32:08,735 - utils - INFO - 1, epoch: 1738, all client loss: [0.5118627548217773, 0.4556356966495514], all pred client disparities: [0.010296285152435303, 0.0022509247064590454], all client disparities: [0.0032608509063720703, 0.010812610387802124], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([0.7357, 0.0000, 0.2643, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:08,980 - utils - INFO - stage3_gradient_single_runtime: 0.006269216537475586
2023-09-28 23:32:08,984 - utils - INFO - 1, epoch: 1739, all client loss: [0.5118382573127747, 0.4556339979171753], all pred client disparities: [0.010297417640686035, 0.0022497624158859253], all client disparities: [0.0032608509063720703, 0.010812610387802124], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([0.7359, 0.0000, 0.2641, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:09,067 - utils - INFO - valid: True, epoch: 1739, loss: [0.5763526558876038, 0.4547637104988098], accuracy: [0.7348066568374634, 0.8012422323226929], mean_accuracy:0.7680244445800781,variance_accuracy:0.033217787742614746, disparity: [0.004545450210571289, 0.005071669816970825], mean_disparity:0.004808560013771057,variance_disparity:0.00026310980319976807, pred_disparity: [0.004428476095199585, 0.0003186464309692383]
2023-09-28 23:32:09,206 - utils - INFO - global_valid: True, epoch: 1739,  global_loss: 0.4561154544353485, global_accuracy: 0.8261222945902155,  global_disparity:0.0018109530210494995, global_pred_disparity: 0.0028445571660995483,
2023-09-28 23:32:09,438 - utils - INFO - stage3_gradient_single_runtime: 0.00635075569152832
2023-09-28 23:32:09,445 - utils - INFO - 1, epoch: 1740, all client loss: [0.5118137001991272, 0.4556323289871216], all pred client disparities: [0.01029863953590393, 0.0022485554218292236], all client disparities: [0.0032608509063720703, 0.010812610387802124], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([0.7362, 0.0000, 0.2638, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:09,698 - utils - INFO - stage3_gradient_single_runtime: 0.006262063980102539
2023-09-28 23:32:09,704 - utils - INFO - 1, epoch: 1741, all client loss: [0.5117892622947693, 0.4556306302547455], all pred client disparities: [0.010299861431121826, 0.002247259020805359], all client disparities: [0.0032608509063720703, 0.010812610387802124], all client accs: [0.7481840252876282, 0.7983389496803284],alphas:tensor([0.7364, 0.0000, 0.2636, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:09,948 - utils - INFO - stage3_gradient_single_runtime: 0.00635528564453125
2023-09-28 23:32:09,953 - utils - INFO - 1, epoch: 1742, all client loss: [0.5117647051811218, 0.4556289613246918], all pred client disparities: [0.010301172733306885, 0.002245992422103882], all client disparities: [0.0032608509063720703, 0.011178046464920044], all client accs: [0.7481840252876282, 0.7983078360557556],alphas:tensor([0.7366, 0.0000, 0.2634, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:10,199 - utils - INFO - stage3_gradient_single_runtime: 0.008607864379882812
2023-09-28 23:32:10,203 - utils - INFO - 1, epoch: 1743, all client loss: [0.5117403864860535, 0.45562732219696045], all pred client disparities: [0.010302424430847168, 0.0022446662187576294], all client disparities: [0.0032608509063720703, 0.011178046464920044], all client accs: [0.7481840252876282, 0.7982767224311829],alphas:tensor([ 7.3682e-01, -6.4960e-17,  2.6318e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:10,479 - utils - INFO - stage3_gradient_single_runtime: 0.006996631622314453
2023-09-28 23:32:10,485 - utils - INFO - 1, epoch: 1744, all client loss: [0.5117159485816956, 0.45562565326690674], all pred client disparities: [0.01030382513999939, 0.0022433102130889893], all client disparities: [0.0032608509063720703, 0.010530680418014526], all client accs: [0.7481840252876282, 0.7982144951820374],alphas:tensor([7.3704e-01, 3.2430e-17, 2.6296e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:10,744 - utils - INFO - stage3_gradient_single_runtime: 0.0065805912017822266
2023-09-28 23:32:10,749 - utils - INFO - 1, epoch: 1745, all client loss: [0.5116915702819824, 0.4556240737438202], all pred client disparities: [0.010305196046829224, 0.0022419244050979614], all client disparities: [0.0032608509063720703, 0.009810209274291992], all client accs: [0.7481840252876282, 0.7982144951820374],alphas:tensor([0.7373, 0.0000, 0.2627, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:10,995 - utils - INFO - stage3_gradient_single_runtime: 0.00681757926940918
2023-09-28 23:32:11,001 - utils - INFO - 1, epoch: 1746, all client loss: [0.5116672515869141, 0.4556223750114441], all pred client disparities: [0.010306596755981445, 0.002240508794784546], all client disparities: [0.0032608509063720703, 0.009810209274291992], all client accs: [0.7481840252876282, 0.7982456088066101],alphas:tensor([7.3747e-01, 3.2332e-17, 2.6253e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:11,249 - utils - INFO - stage3_gradient_single_runtime: 0.00653076171875
2023-09-28 23:32:11,254 - utils - INFO - 1, epoch: 1747, all client loss: [0.5116429328918457, 0.45562079548835754], all pred client disparities: [0.010308057069778442, 0.002239048480987549], all client disparities: [0.0032608509063720703, 0.009883299469947815], all client accs: [0.7481840252876282, 0.7982767224311829],alphas:tensor([7.3768e-01, 3.2283e-17, 2.6232e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:11,499 - utils - INFO - stage3_gradient_single_runtime: 0.0064601898193359375
2023-09-28 23:32:11,504 - utils - INFO - 1, epoch: 1748, all client loss: [0.5116186738014221, 0.455619215965271], all pred client disparities: [0.010309487581253052, 0.002237573266029358], all client disparities: [0.0032608509063720703, 0.009883299469947815], all client accs: [0.7481840252876282, 0.7982767224311829],alphas:tensor([7.3789e-01, 3.2235e-17, 2.6211e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:11,751 - utils - INFO - stage3_gradient_single_runtime: 0.006551027297973633
2023-09-28 23:32:11,756 - utils - INFO - 1, epoch: 1749, all client loss: [0.5115944743156433, 0.45561760663986206], all pred client disparities: [0.010310977697372437, 0.0022360235452651978], all client disparities: [0.0032608509063720703, 0.009956389665603638], all client accs: [0.7481840252876282, 0.7983078360557556],alphas:tensor([7.3810e-01, 3.2187e-17, 2.6190e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:11,995 - utils - INFO - stage3_gradient_single_runtime: 0.006453275680541992
2023-09-28 23:32:12,000 - utils - INFO - 1, epoch: 1750, all client loss: [0.5115702748298645, 0.4556160271167755], all pred client disparities: [0.010312587022781372, 0.002234533429145813], all client disparities: [0.0032608509063720703, 0.009956389665603638], all client accs: [0.7481840252876282, 0.7983078360557556],alphas:tensor([0.7383, 0.0000, 0.2617, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:12,240 - utils - INFO - stage3_gradient_single_runtime: 0.006457805633544922
2023-09-28 23:32:12,245 - utils - INFO - 1, epoch: 1751, all client loss: [0.5115460753440857, 0.45561450719833374], all pred client disparities: [0.010314136743545532, 0.0022329092025756836], all client disparities: [0.0032608509063720703, 0.00973711907863617], all client accs: [0.7481840252876282, 0.7988055348396301],alphas:tensor([0.7385, 0.0000, 0.2615, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:12,486 - utils - INFO - stage3_gradient_single_runtime: 0.006427288055419922
2023-09-28 23:32:12,491 - utils - INFO - 1, epoch: 1752, all client loss: [0.5115219354629517, 0.4556129574775696], all pred client disparities: [0.010315656661987305, 0.002231284976005554], all client disparities: [0.0032608509063720703, 0.00973711907863617], all client accs: [0.7481840252876282, 0.7987744212150574],alphas:tensor([0.7387, 0.0000, 0.2613, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:12,684 - utils - INFO - stage3_gradient_single_runtime: 0.007570505142211914
2023-09-28 23:32:12,690 - utils - INFO - 1, epoch: 1753, all client loss: [0.5114977955818176, 0.4556114673614502], all pred client disparities: [0.010317355394363403, 0.002229630947113037], all client disparities: [0.0032608509063720703, 0.00973711907863617], all client accs: [0.7481840252876282, 0.7986810803413391],alphas:tensor([0.7389, 0.0000, 0.2611, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:12,940 - utils - INFO - stage3_gradient_single_runtime: 0.007001161575317383
2023-09-28 23:32:12,946 - utils - INFO - 1, epoch: 1754, all client loss: [0.5114737153053284, 0.4556099474430084], all pred client disparities: [0.010319054126739502, 0.002227962017059326], all client disparities: [0.0032608509063720703, 0.00973711907863617], all client accs: [0.7481840252876282, 0.7986810803413391],alphas:tensor([0.7391, 0.0000, 0.2609, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:13,190 - utils - INFO - stage3_gradient_single_runtime: 0.00753331184387207
2023-09-28 23:32:13,196 - utils - INFO - 1, epoch: 1755, all client loss: [0.5114496350288391, 0.45560845732688904], all pred client disparities: [0.0103207528591156, 0.0022262483835220337], all client disparities: [0.0032608509063720703, 0.00973711907863617], all client accs: [0.7481840252876282, 0.7986810803413391],alphas:tensor([0.7393, 0.0000, 0.2607, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:13,444 - utils - INFO - stage3_gradient_single_runtime: 0.006314754486083984
2023-09-28 23:32:13,450 - utils - INFO - 1, epoch: 1756, all client loss: [0.5114256143569946, 0.45560699701309204], all pred client disparities: [0.010322481393814087, 0.002224549651145935], all client disparities: [0.0032608509063720703, 0.009089753031730652], all client accs: [0.7481840252876282, 0.7985877990722656],alphas:tensor([0.7395, 0.0000, 0.2605, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:13,687 - utils - INFO - stage3_gradient_single_runtime: 0.007357120513916016
2023-09-28 23:32:13,690 - utils - INFO - 1, epoch: 1757, all client loss: [0.5114016532897949, 0.45560553669929504], all pred client disparities: [0.010324239730834961, 0.002222776412963867], all client disparities: [0.0032608509063720703, 0.009089753031730652], all client accs: [0.7481840252876282, 0.7985877990722656],alphas:tensor([0.7397, 0.0000, 0.2603, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:13,952 - utils - INFO - stage3_gradient_single_runtime: 0.009876489639282227
2023-09-28 23:32:13,958 - utils - INFO - 1, epoch: 1758, all client loss: [0.5113776922225952, 0.45560410618782043], all pred client disparities: [0.010325998067855835, 0.0022209733724594116], all client disparities: [0.0032608509063720703, 0.009089753031730652], all client accs: [0.7481840252876282, 0.7985566854476929],alphas:tensor([0.7399, 0.0000, 0.2601, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:14,212 - utils - INFO - stage3_gradient_single_runtime: 0.006342172622680664
2023-09-28 23:32:14,216 - utils - INFO - 1, epoch: 1759, all client loss: [0.5113537311553955, 0.45560264587402344], all pred client disparities: [0.010327816009521484, 0.0022191405296325684], all client disparities: [0.0032608509063720703, 0.009089753031730652], all client accs: [0.7481840252876282, 0.7985566854476929],alphas:tensor([ 7.4010e-01, -6.3456e-17,  2.5990e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:14,349 - utils - INFO - valid: True, epoch: 1759, loss: [0.5762572288513184, 0.45472389459609985], accuracy: [0.7348066568374634, 0.8018012642860413], mean_accuracy:0.7683039605617523,variance_accuracy:0.03349730372428894, disparity: [0.004545450210571289, 0.005515456199645996], mean_disparity:0.005030453205108643,variance_disparity:0.0004850029945373535, pred_disparity: [0.004429131746292114, 8.463859558105469e-06]
2023-09-28 23:32:14,434 - utils - INFO - global_valid: True, epoch: 1759,  global_loss: 0.4560750424861908, global_accuracy: 0.8262626709330536,  global_disparity:0.0022407472133636475, global_pred_disparity: 0.003138601779937744,
2023-09-28 23:32:14,676 - utils - INFO - stage3_gradient_single_runtime: 0.006469011306762695
2023-09-28 23:32:14,681 - utils - INFO - 1, epoch: 1760, all client loss: [0.5113298892974854, 0.4556012451648712], all pred client disparities: [0.010329663753509521, 0.0022172480821609497], all client disparities: [0.0032608509063720703, 0.009089753031730652], all client accs: [0.7481840252876282, 0.7985255718231201],alphas:tensor([ 7.4029e-01, -6.3368e-17,  2.5971e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:14,925 - utils - INFO - stage3_gradient_single_runtime: 0.006388664245605469
2023-09-28 23:32:14,927 - utils - INFO - 1, epoch: 1761, all client loss: [0.5113059878349304, 0.4555998742580414], all pred client disparities: [0.010331481695175171, 0.0022154301404953003], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7985566854476929],alphas:tensor([ 7.4048e-01, -6.3281e-17,  2.5952e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:15,181 - utils - INFO - stage3_gradient_single_runtime: 0.0063512325286865234
2023-09-28 23:32:15,186 - utils - INFO - 1, epoch: 1762, all client loss: [0.5112821459770203, 0.45559847354888916], all pred client disparities: [0.010333448648452759, 0.0022134780883789062], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7985566854476929],alphas:tensor([ 7.4066e-01, -6.3194e-17,  2.5934e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:15,439 - utils - INFO - stage3_gradient_single_runtime: 0.006367683410644531
2023-09-28 23:32:15,445 - utils - INFO - 1, epoch: 1763, all client loss: [0.5112583637237549, 0.4555971324443817], all pred client disparities: [0.010335296392440796, 0.002211540937423706], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7985566854476929],alphas:tensor([0.7408, 0.0000, 0.2592, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:15,689 - utils - INFO - stage3_gradient_single_runtime: 0.00695037841796875
2023-09-28 23:32:15,694 - utils - INFO - 1, epoch: 1764, all client loss: [0.5112346410751343, 0.45559582114219666], all pred client disparities: [0.01033732295036316, 0.002209603786468506], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7985566854476929],alphas:tensor([0.7410, 0.0000, 0.2590, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:15,942 - utils - INFO - stage3_gradient_single_runtime: 0.0072422027587890625
2023-09-28 23:32:15,947 - utils - INFO - 1, epoch: 1765, all client loss: [0.5112108588218689, 0.4555945098400116], all pred client disparities: [0.010339349508285522, 0.0022075921297073364], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7985255718231201],alphas:tensor([ 7.4122e-01, -6.2938e-17,  2.5878e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:16,198 - utils - INFO - stage3_gradient_single_runtime: 0.0063512325286865234
2023-09-28 23:32:16,204 - utils - INFO - 1, epoch: 1766, all client loss: [0.5111871361732483, 0.45559319853782654], all pred client disparities: [0.01034131646156311, 0.002205595374107361], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7985255718231201],alphas:tensor([0.7414, 0.0000, 0.2586, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:16,449 - utils - INFO - stage3_gradient_single_runtime: 0.006349325180053711
2023-09-28 23:32:16,454 - utils - INFO - 1, epoch: 1767, all client loss: [0.5111634731292725, 0.45559191703796387], all pred client disparities: [0.010343372821807861, 0.002203524112701416], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7985255718231201],alphas:tensor([0.7416, 0.0000, 0.2584, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:16,692 - utils - INFO - stage3_gradient_single_runtime: 0.0063822269439697266
2023-09-28 23:32:16,698 - utils - INFO - 1, epoch: 1768, all client loss: [0.5111398696899414, 0.4555906355381012], all pred client disparities: [0.010345399379730225, 0.002201467752456665], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7983700633049011],alphas:tensor([ 7.4176e-01, -6.2687e-17,  2.5824e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:16,952 - utils - INFO - stage3_gradient_single_runtime: 0.007382631301879883
2023-09-28 23:32:16,957 - utils - INFO - 1, epoch: 1769, all client loss: [0.5111162066459656, 0.4555894434452057], all pred client disparities: [0.010347515344619751, 0.0021993666887283325], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7983700633049011],alphas:tensor([0.7419, 0.0000, 0.2581, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:17,211 - utils - INFO - stage3_gradient_single_runtime: 0.0063359737396240234
2023-09-28 23:32:17,216 - utils - INFO - 1, epoch: 1770, all client loss: [0.5110926628112793, 0.4555882215499878], all pred client disparities: [0.010349631309509277, 0.0021972358226776123], all client disparities: [0.0032608509063720703, 0.008724302053451538], all client accs: [0.7481840252876282, 0.7983700633049011],alphas:tensor([0.7421, 0.0000, 0.2579, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:17,460 - utils - INFO - stage3_gradient_single_runtime: 0.006955146789550781
2023-09-28 23:32:17,462 - utils - INFO - 1, epoch: 1771, all client loss: [0.511069118976593, 0.4555869996547699], all pred client disparities: [0.010351777076721191, 0.0021950751543045044], all client disparities: [0.0032608509063720703, 0.007648825645446777], all client accs: [0.7481840252876282, 0.7975924015045166],alphas:tensor([ 7.4228e-01, -6.2442e-17,  2.5772e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:17,712 - utils - INFO - stage3_gradient_single_runtime: 0.007297992706298828
2023-09-28 23:32:17,717 - utils - INFO - 1, epoch: 1772, all client loss: [0.5110456347465515, 0.4555858373641968], all pred client disparities: [0.010354012250900269, 0.0021928995847702026], all client disparities: [0.0032608509063720703, 0.007648825645446777], all client accs: [0.7481840252876282, 0.7975924015045166],alphas:tensor([ 7.4246e-01, -6.2362e-17,  2.5754e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:17,966 - utils - INFO - stage3_gradient_single_runtime: 0.007258892059326172
2023-09-28 23:32:17,972 - utils - INFO - 1, epoch: 1773, all client loss: [0.5110220909118652, 0.45558470487594604], all pred client disparities: [0.010356098413467407, 0.002190724015235901], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7481840252876282, 0.7975924015045166],alphas:tensor([7.4263e-01, 3.1141e-17, 2.5737e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:18,214 - utils - INFO - stage3_gradient_single_runtime: 0.0065135955810546875
2023-09-28 23:32:18,218 - utils - INFO - 1, epoch: 1774, all client loss: [0.5109986662864685, 0.4555835723876953], all pred client disparities: [0.010358363389968872, 0.0021884888410568237], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7481840252876282, 0.7975924015045166],alphas:tensor([7.4280e-01, 3.1101e-17, 2.5720e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:18,457 - utils - INFO - stage3_gradient_single_runtime: 0.006533622741699219
2023-09-28 23:32:18,461 - utils - INFO - 1, epoch: 1775, all client loss: [0.5109752416610718, 0.4555824398994446], all pred client disparities: [0.010360628366470337, 0.0021862387657165527], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7481840252876282, 0.7975612878799438],alphas:tensor([0.7430, 0.0000, 0.2570, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:18,698 - utils - INFO - stage3_gradient_single_runtime: 0.0065343379974365234
2023-09-28 23:32:18,703 - utils - INFO - 1, epoch: 1776, all client loss: [0.510951817035675, 0.45558133721351624], all pred client disparities: [0.010362893342971802, 0.0021839439868927], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7481840252876282, 0.7975612878799438],alphas:tensor([7.4313e-01, 3.1023e-17, 2.5687e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:18,942 - utils - INFO - stage3_gradient_single_runtime: 0.006486415863037109
2023-09-28 23:32:18,947 - utils - INFO - 1, epoch: 1777, all client loss: [0.5109285116195679, 0.4555802643299103], all pred client disparities: [0.01036524772644043, 0.00218161940574646], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([0.7433, 0.0000, 0.2567, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:19,187 - utils - INFO - stage3_gradient_single_runtime: 0.006491661071777344
2023-09-28 23:32:19,193 - utils - INFO - 1, epoch: 1778, all client loss: [0.5109052062034607, 0.4555792212486267], all pred client disparities: [0.010367542505264282, 0.0021793246269226074], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7481840252876282, 0.7975301742553711],alphas:tensor([0.7435, 0.0000, 0.2565, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:19,434 - utils - INFO - stage3_gradient_single_runtime: 0.006464242935180664
2023-09-28 23:32:19,438 - utils - INFO - 1, epoch: 1779, all client loss: [0.5108819007873535, 0.45557817816734314], all pred client disparities: [0.010369837284088135, 0.0021769702434539795], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7481840252876282, 0.7974990606307983],alphas:tensor([0.7436, 0.0000, 0.2564, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:19,521 - utils - INFO - valid: True, epoch: 1779, loss: [0.5761569738388062, 0.4546911418437958], accuracy: [0.7348066568374634, 0.801118016242981], mean_accuracy:0.7679623365402222,variance_accuracy:0.03315567970275879, disparity: [0.004545450210571289, 0.004197105765342712], mean_disparity:0.004371277987957001,variance_disparity:0.00017417222261428833, pred_disparity: [0.004440516233444214, 0.00036053359508514404]
2023-09-28 23:32:19,655 - utils - INFO - global_valid: True, epoch: 1779,  global_loss: 0.45604151487350464, global_accuracy: 0.8265981021227179,  global_disparity:0.0009795278310775757, global_pred_disparity: 0.003457680344581604,
2023-09-28 23:32:19,899 - utils - INFO - stage3_gradient_single_runtime: 0.006509065628051758
2023-09-28 23:32:19,904 - utils - INFO - 1, epoch: 1780, all client loss: [0.5108586549758911, 0.45557722449302673], all pred client disparities: [0.010372281074523926, 0.002174586057662964], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7481840252876282, 0.7974990606307983],alphas:tensor([0.7438, 0.0000, 0.2562, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:20,149 - utils - INFO - stage3_gradient_single_runtime: 0.0065729618072509766
2023-09-28 23:32:20,154 - utils - INFO - 1, epoch: 1781, all client loss: [0.5108354687690735, 0.45557624101638794], all pred client disparities: [0.010374605655670166, 0.0021722018718719482], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7974679470062256],alphas:tensor([7.4394e-01, 3.0832e-17, 2.5606e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:20,397 - utils - INFO - stage3_gradient_single_runtime: 0.0064373016357421875
2023-09-28 23:32:20,402 - utils - INFO - 1, epoch: 1782, all client loss: [0.5108122825622559, 0.45557528734207153], all pred client disparities: [0.010377079248428345, 0.002169772982597351], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7974679470062256],alphas:tensor([7.4410e-01, 3.0795e-17, 2.5590e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:20,644 - utils - INFO - stage3_gradient_single_runtime: 0.006483316421508789
2023-09-28 23:32:20,649 - utils - INFO - 1, epoch: 1783, all client loss: [0.5107890963554382, 0.45557430386543274], all pred client disparities: [0.010379493236541748, 0.0021673589944839478], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7974679470062256],alphas:tensor([7.4426e-01, 3.0758e-17, 2.5574e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:20,890 - utils - INFO - stage3_gradient_single_runtime: 0.006509542465209961
2023-09-28 23:32:20,895 - utils - INFO - 1, epoch: 1784, all client loss: [0.5107659101486206, 0.4555733799934387], all pred client disparities: [0.010381937026977539, 0.002164870500564575], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7974679470062256],alphas:tensor([0.7444, 0.0000, 0.2556, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:21,137 - utils - INFO - stage3_gradient_single_runtime: 0.006447792053222656
2023-09-28 23:32:21,142 - utils - INFO - 1, epoch: 1785, all client loss: [0.5107429027557373, 0.4555725157260895], all pred client disparities: [0.010384410619735718, 0.0021623969078063965], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7974679470062256],alphas:tensor([0.7446, 0.0000, 0.2554, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:21,382 - utils - INFO - stage3_gradient_single_runtime: 0.006464958190917969
2023-09-28 23:32:21,387 - utils - INFO - 1, epoch: 1786, all client loss: [0.5107198357582092, 0.45557162165641785], all pred client disparities: [0.01038697361946106, 0.0021598637104034424], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7974368333816528],alphas:tensor([ 7.4472e-01, -6.1297e-17,  2.5528e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:21,630 - utils - INFO - stage3_gradient_single_runtime: 0.006566047668457031
2023-09-28 23:32:21,635 - utils - INFO - 1, epoch: 1787, all client loss: [0.5106967687606812, 0.455570787191391], all pred client disparities: [0.010389447212219238, 0.002157345414161682], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7974057197570801],alphas:tensor([0.7449, 0.0000, 0.2551, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:21,880 - utils - INFO - stage3_gradient_single_runtime: 0.0066220760345458984
2023-09-28 23:32:21,885 - utils - INFO - 1, epoch: 1788, all client loss: [0.5106737613677979, 0.45556995272636414], all pred client disparities: [0.010391950607299805, 0.0021547824144363403], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7973746657371521],alphas:tensor([0.7450, 0.0000, 0.2550, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:22,140 - utils - INFO - stage3_gradient_single_runtime: 0.0067064762115478516
2023-09-28 23:32:22,145 - utils - INFO - 1, epoch: 1789, all client loss: [0.5106508135795593, 0.45556917786598206], all pred client disparities: [0.010394662618637085, 0.0021522343158721924], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7973746657371521],alphas:tensor([0.7452, 0.0000, 0.2548, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:22,387 - utils - INFO - stage3_gradient_single_runtime: 0.0064678192138671875
2023-09-28 23:32:22,392 - utils - INFO - 1, epoch: 1790, all client loss: [0.5106279253959656, 0.4555684030056], all pred client disparities: [0.010397195816040039, 0.0021495968103408813], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7972813248634338],alphas:tensor([7.4532e-01, 3.0507e-17, 2.5468e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:22,633 - utils - INFO - stage3_gradient_single_runtime: 0.0064504146575927734
2023-09-28 23:32:22,638 - utils - INFO - 1, epoch: 1791, all client loss: [0.510604977607727, 0.4555676281452179], all pred client disparities: [0.010399848222732544, 0.002146989107131958], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7972190976142883],alphas:tensor([7.4547e-01, 3.0472e-17, 2.5453e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:22,879 - utils - INFO - stage3_gradient_single_runtime: 0.0065004825592041016
2023-09-28 23:32:22,884 - utils - INFO - 1, epoch: 1792, all client loss: [0.5105821490287781, 0.4555668830871582], all pred client disparities: [0.010402470827102661, 0.002144366502761841], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7970635890960693],alphas:tensor([ 7.4562e-01, -6.0875e-17,  2.5438e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:23,123 - utils - INFO - stage3_gradient_single_runtime: 0.006560802459716797
2023-09-28 23:32:23,128 - utils - INFO - 1, epoch: 1793, all client loss: [0.5105593204498291, 0.4555661976337433], all pred client disparities: [0.010405123233795166, 0.0021416395902633667], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7970635890960693],alphas:tensor([0.7458, 0.0000, 0.2542, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:23,368 - utils - INFO - stage3_gradient_single_runtime: 0.00657963752746582
2023-09-28 23:32:23,373 - utils - INFO - 1, epoch: 1794, all client loss: [0.5105365514755249, 0.45556551218032837], all pred client disparities: [0.010407805442810059, 0.0021390020847320557], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7970324754714966],alphas:tensor([7.4590e-01, 3.0369e-17, 2.5410e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:23,615 - utils - INFO - stage3_gradient_single_runtime: 0.0065228939056396484
2023-09-28 23:32:23,620 - utils - INFO - 1, epoch: 1795, all client loss: [0.5105137825012207, 0.4555648863315582], all pred client disparities: [0.010410547256469727, 0.0021362751722335815], all client disparities: [0.0032608509063720703, 0.006928369402885437], all client accs: [0.7506053447723389, 0.7969702482223511],alphas:tensor([0.7460, 0.0000, 0.2540, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:23,862 - utils - INFO - stage3_gradient_single_runtime: 0.006482124328613281
2023-09-28 23:32:23,867 - utils - INFO - 1, epoch: 1796, all client loss: [0.5104910731315613, 0.4555642604827881], all pred client disparities: [0.010413259267807007, 0.0021335631608963013], all client disparities: [0.0032608509063720703, 0.0054874420166015625], all client accs: [0.7506053447723389, 0.7970013618469238],alphas:tensor([0.7462, 0.0000, 0.2538, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:24,109 - utils - INFO - stage3_gradient_single_runtime: 0.006475687026977539
2023-09-28 23:32:24,114 - utils - INFO - 1, epoch: 1797, all client loss: [0.5104683637619019, 0.45556360483169556], all pred client disparities: [0.010416030883789062, 0.0021308213472366333], all client disparities: [0.0032608509063720703, 0.0054874420166015625], all client accs: [0.7506053447723389, 0.7970013618469238],alphas:tensor([7.4633e-01, 3.0269e-17, 2.5367e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:24,356 - utils - INFO - stage3_gradient_single_runtime: 0.006486177444458008
2023-09-28 23:32:24,362 - utils - INFO - 1, epoch: 1798, all client loss: [0.5104456543922424, 0.4555630385875702], all pred client disparities: [0.010418802499771118, 0.0021280646324157715], all client disparities: [0.0032608509063720703, 0.0054874420166015625], all client accs: [0.7506053447723389, 0.7970013618469238],alphas:tensor([0.7465, 0.0000, 0.2535, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:24,602 - utils - INFO - stage3_gradient_single_runtime: 0.006548643112182617
2023-09-28 23:32:24,607 - utils - INFO - 1, epoch: 1799, all client loss: [0.5104230642318726, 0.4555625021457672], all pred client disparities: [0.01042163372039795, 0.0021252483129501343], all client disparities: [0.0032608509063720703, 0.0054874420166015625], all client accs: [0.7506053447723389, 0.7970013618469238],alphas:tensor([0.7466, 0.0000, 0.2534, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:24,689 - utils - INFO - valid: True, epoch: 1799, loss: [0.5760551691055298, 0.4546672999858856], accuracy: [0.7403315305709839, 0.8009316921234131], mean_accuracy:0.7706316113471985,variance_accuracy:0.0303000807762146, disparity: [0.004545450210571289, 0.004345044493675232], mean_disparity:0.0044452473521232605,variance_disparity:0.00010020285844802856, pred_disparity: [0.004461765289306641, 0.0007314980030059814]
2023-09-28 23:32:24,819 - utils - INFO - global_valid: True, epoch: 1799,  global_loss: 0.4560168385505676, global_accuracy: 0.8268849917647692,  global_disparity:0.001122787594795227, global_pred_disparity: 0.0037960857152938843,
2023-09-28 23:32:25,069 - utils - INFO - stage3_gradient_single_runtime: 0.0065991878509521484
2023-09-28 23:32:25,074 - utils - INFO - 1, epoch: 1800, all client loss: [0.5104004740715027, 0.45556196570396423], all pred client disparities: [0.010424405336380005, 0.002122431993484497], all client disparities: [0.0032608509063720703, 0.0054874420166015625], all client accs: [0.7506053447723389, 0.7970013618469238],alphas:tensor([7.4674e-01, 3.0170e-17, 2.5326e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:25,310 - utils - INFO - stage3_gradient_single_runtime: 0.00641179084777832
2023-09-28 23:32:25,312 - utils - INFO - 1, epoch: 1801, all client loss: [0.5103779435157776, 0.45556145906448364], all pred client disparities: [0.010427236557006836, 0.00211961567401886], all client disparities: [0.0032608509063720703, 0.0054874420166015625], all client accs: [0.7506053447723389, 0.7970013618469238],alphas:tensor([0.7469, 0.0000, 0.2531, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:25,549 - utils - INFO - stage3_gradient_single_runtime: 0.008341550827026367
2023-09-28 23:32:25,554 - utils - INFO - 1, epoch: 1802, all client loss: [0.5103554129600525, 0.45556098222732544], all pred client disparities: [0.010430067777633667, 0.002116754651069641], all client disparities: [0.0032608509063720703, 0.005633622407913208], all client accs: [0.7506053447723389, 0.7970324754714966],alphas:tensor([7.4701e-01, 3.0106e-17, 2.5299e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:25,796 - utils - INFO - stage3_gradient_single_runtime: 0.006646633148193359
2023-09-28 23:32:25,801 - utils - INFO - 1, epoch: 1803, all client loss: [0.5103328824043274, 0.45556047558784485], all pred client disparities: [0.010432958602905273, 0.0021138936281204224], all client disparities: [0.0032608509063720703, 0.005633622407913208], all client accs: [0.7506053447723389, 0.7969702482223511],alphas:tensor([7.4714e-01, 3.0074e-17, 2.5286e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:25,984 - utils - INFO - stage3_gradient_single_runtime: 0.006407022476196289
2023-09-28 23:32:25,989 - utils - INFO - 1, epoch: 1804, all client loss: [0.5103104710578918, 0.4555600583553314], all pred client disparities: [0.01043584942817688, 0.002111002802848816], all client disparities: [0.0032608509063720703, 0.005633622407913208], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([ 7.4727e-01, -6.0085e-17,  2.5273e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:26,239 - utils - INFO - stage3_gradient_single_runtime: 0.006307125091552734
2023-09-28 23:32:26,244 - utils - INFO - 1, epoch: 1805, all client loss: [0.5102880001068115, 0.4555596709251404], all pred client disparities: [0.010438740253448486, 0.0021080821752548218], all client disparities: [0.0032608509063720703, 0.005633622407913208], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([ 7.4740e-01, -6.0023e-17,  2.5260e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:26,470 - utils - INFO - stage3_gradient_single_runtime: 0.0063669681549072266
2023-09-28 23:32:26,473 - utils - INFO - 1, epoch: 1806, all client loss: [0.510265588760376, 0.45555925369262695], all pred client disparities: [0.010441690683364868, 0.0021051615476608276], all client disparities: [0.0032608509063720703, 0.005633622407913208], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([7.4753e-01, 2.9980e-17, 2.5247e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:26,702 - utils - INFO - stage3_gradient_single_runtime: 0.0063250064849853516
2023-09-28 23:32:26,707 - utils - INFO - 1, epoch: 1807, all client loss: [0.51024329662323, 0.4555588960647583], all pred client disparities: [0.01044464111328125, 0.002102196216583252], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7969080209732056],alphas:tensor([7.4766e-01, 2.9949e-17, 2.5234e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:26,937 - utils - INFO - stage3_gradient_single_runtime: 0.0063266754150390625
2023-09-28 23:32:26,941 - utils - INFO - 1, epoch: 1808, all client loss: [0.5102208852767944, 0.45555856823921204], all pred client disparities: [0.010447591543197632, 0.00209924578666687], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7969080209732056],alphas:tensor([0.7478, 0.0000, 0.2522, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:27,170 - utils - INFO - stage3_gradient_single_runtime: 0.0062634944915771484
2023-09-28 23:32:27,175 - utils - INFO - 1, epoch: 1809, all client loss: [0.5101986527442932, 0.45555827021598816], all pred client disparities: [0.010450601577758789, 0.0020962804555892944], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([0.7479, 0.0000, 0.2521, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:27,404 - utils - INFO - stage3_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:32:27,409 - utils - INFO - 1, epoch: 1810, all client loss: [0.5101763606071472, 0.45555800199508667], all pred client disparities: [0.010453611612319946, 0.002093270421028137], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([ 7.4804e-01, -5.9717e-17,  2.5196e-01,  0.0000e+00,  0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:27,639 - utils - INFO - stage3_gradient_single_runtime: 0.006285429000854492
2023-09-28 23:32:27,644 - utils - INFO - 1, epoch: 1811, all client loss: [0.510154128074646, 0.4555577039718628], all pred client disparities: [0.010456591844558716, 0.00209026038646698], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([0.7482, 0.0000, 0.2518, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:27,884 - utils - INFO - stage3_gradient_single_runtime: 0.0063381195068359375
2023-09-28 23:32:27,889 - utils - INFO - 1, epoch: 1812, all client loss: [0.5101319551467896, 0.4555574953556061], all pred client disparities: [0.01045963168144226, 0.0020871907472610474], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([0.7483, 0.0000, 0.2517, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:28,118 - utils - INFO - stage3_gradient_single_runtime: 0.007101297378540039
2023-09-28 23:32:28,123 - utils - INFO - 1, epoch: 1813, all client loss: [0.5101097822189331, 0.45555728673934937], all pred client disparities: [0.010462731122970581, 0.0020841211080551147], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([0.7484, 0.0000, 0.2516, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:28,360 - utils - INFO - stage3_gradient_single_runtime: 0.006683826446533203
2023-09-28 23:32:28,365 - utils - INFO - 1, epoch: 1814, all client loss: [0.5100876092910767, 0.45555710792541504], all pred client disparities: [0.010465770959854126, 0.00208108127117157], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([0.7485, 0.0000, 0.2515, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:28,607 - utils - INFO - stage3_gradient_single_runtime: 0.0062563419342041016
2023-09-28 23:32:28,612 - utils - INFO - 1, epoch: 1815, all client loss: [0.5100655555725098, 0.4555569589138031], all pred client disparities: [0.010468900203704834, 0.0020779818296432495], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([0.7486, 0.0000, 0.2514, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:28,838 - utils - INFO - stage3_gradient_single_runtime: 0.006305217742919922
2023-09-28 23:32:28,843 - utils - INFO - 1, epoch: 1816, all client loss: [0.5100435018539429, 0.45555680990219116], all pred client disparities: [0.010471969842910767, 0.002074897289276123], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([7.4876e-01, 2.9682e-17, 2.5124e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:29,069 - utils - INFO - stage3_gradient_single_runtime: 0.006300210952758789
2023-09-28 23:32:29,074 - utils - INFO - 1, epoch: 1817, all client loss: [0.510021448135376, 0.4555567502975464], all pred client disparities: [0.010475218296051025, 0.0020717233419418335], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7969080209732056],alphas:tensor([0.7489, 0.0000, 0.2511, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:29,303 - utils - INFO - stage3_gradient_single_runtime: 0.006311178207397461
2023-09-28 23:32:29,308 - utils - INFO - 1, epoch: 1818, all client loss: [0.5099994540214539, 0.4555566608905792], all pred client disparities: [0.010478287935256958, 0.0020685940980911255], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7969080209732056],alphas:tensor([0.7490, 0.0000, 0.2510, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:29,535 - utils - INFO - stage3_gradient_single_runtime: 0.006253480911254883
2023-09-28 23:32:29,540 - utils - INFO - 1, epoch: 1819, all client loss: [0.5099775195121765, 0.45555663108825684], all pred client disparities: [0.010481506586074829, 0.002065405249595642], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7969080209732056],alphas:tensor([0.7491, 0.0000, 0.2509, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:29,666 - utils - INFO - valid: True, epoch: 1819, loss: [0.5759544968605042, 0.4546535611152649], accuracy: [0.7403315305709839, 0.8003105521202087], mean_accuracy:0.7703210413455963,variance_accuracy:0.029989510774612427, disparity: [0.004545450210571289, 0.0028787702322006226], mean_disparity:0.003712110221385956,variance_disparity:0.0008333399891853333, pred_disparity: [0.004491657018661499, 0.0011162012815475464]
2023-09-28 23:32:29,742 - utils - INFO - global_valid: True, epoch: 1819,  global_loss: 0.4560021162033081, global_accuracy: 0.8272050439717629,  global_disparity:0.00028170645236968994, global_pred_disparity: 0.004148751497268677,
2023-09-28 23:32:29,971 - utils - INFO - stage3_gradient_single_runtime: 0.006319284439086914
2023-09-28 23:32:29,976 - utils - INFO - 1, epoch: 1820, all client loss: [0.5099555850028992, 0.45555657148361206], all pred client disparities: [0.010484665632247925, 0.0020622462034225464], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7969080209732056],alphas:tensor([0.7492, 0.0000, 0.2508, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:30,204 - utils - INFO - stage3_gradient_single_runtime: 0.006264448165893555
2023-09-28 23:32:30,209 - utils - INFO - 1, epoch: 1821, all client loss: [0.5099336504936218, 0.45555657148361206], all pred client disparities: [0.010487884283065796, 0.0020590275526046753], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7969080209732056],alphas:tensor([0.7493, 0.0000, 0.2507, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:30,438 - utils - INFO - stage3_gradient_single_runtime: 0.00627899169921875
2023-09-28 23:32:30,443 - utils - INFO - 1, epoch: 1822, all client loss: [0.509911835193634, 0.45555660128593445], all pred client disparities: [0.010491102933883667, 0.002055808901786804], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7969080209732056],alphas:tensor([0.7495, 0.0000, 0.2505, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:30,673 - utils - INFO - stage3_gradient_single_runtime: 0.006319999694824219
2023-09-28 23:32:30,678 - utils - INFO - 1, epoch: 1823, all client loss: [0.5098900198936462, 0.4555566608905792], all pred client disparities: [0.010494321584701538, 0.002052605152130127], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968458533287048],alphas:tensor([0.7496, 0.0000, 0.2504, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:30,905 - utils - INFO - stage3_gradient_single_runtime: 0.0062656402587890625
2023-09-28 23:32:30,909 - utils - INFO - 1, epoch: 1824, all client loss: [0.5098681449890137, 0.455556720495224], all pred client disparities: [0.010497599840164185, 0.0020493119955062866], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968458533287048],alphas:tensor([7.4968e-01, 2.9460e-17, 2.5032e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:31,139 - utils - INFO - stage3_gradient_single_runtime: 0.006280422210693359
2023-09-28 23:32:31,144 - utils - INFO - 1, epoch: 1825, all client loss: [0.5098463892936707, 0.45555683970451355], all pred client disparities: [0.010500907897949219, 0.002046063542366028], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7968458533287048],alphas:tensor([7.4979e-01, 2.9433e-17, 2.5021e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:31,371 - utils - INFO - stage3_gradient_single_runtime: 0.006288766860961914
2023-09-28 23:32:31,376 - utils - INFO - 1, epoch: 1826, all client loss: [0.5098246932029724, 0.4555569589138031], all pred client disparities: [0.01050412654876709, 0.0020427703857421875], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7967836260795593],alphas:tensor([0.7499, 0.0000, 0.2501, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:31,604 - utils - INFO - stage3_gradient_single_runtime: 0.006299257278442383
2023-09-28 23:32:31,608 - utils - INFO - 1, epoch: 1827, all client loss: [0.5098029971122742, 0.45555710792541504], all pred client disparities: [0.010507464408874512, 0.002039477229118347], all client disparities: [0.0032608509063720703, 0.005132421851158142], all client accs: [0.7506053447723389, 0.7967836260795593],alphas:tensor([0.7500, 0.0000, 0.2500, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:31,835 - utils - INFO - stage3_gradient_single_runtime: 0.006318330764770508
2023-09-28 23:32:31,840 - utils - INFO - 1, epoch: 1828, all client loss: [0.5097813606262207, 0.45555728673934937], all pred client disparities: [0.010510861873626709, 0.0020361393690109253], all client disparities: [0.0032608509063720703, 0.004411965608596802], all client accs: [0.7506053447723389, 0.7968147397041321],alphas:tensor([7.5011e-01, 2.9353e-17, 2.4989e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:32,069 - utils - INFO - stage3_gradient_single_runtime: 0.006246805191040039
2023-09-28 23:32:32,073 - utils - INFO - 1, epoch: 1829, all client loss: [0.5097597241401672, 0.4555574953556061], all pred client disparities: [0.010514140129089355, 0.0020328015089035034], all client disparities: [0.0032608509063720703, 0.004411965608596802], all client accs: [0.7506053447723389, 0.7967836260795593],alphas:tensor([0.7502, 0.0000, 0.2498, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:32,302 - utils - INFO - stage3_gradient_single_runtime: 0.006251335144042969
2023-09-28 23:32:32,307 - utils - INFO - 1, epoch: 1830, all client loss: [0.5097381472587585, 0.4555577039718628], all pred client disparities: [0.010517537593841553, 0.0020294785499572754], all client disparities: [0.0032608509063720703, 0.003701925277709961], all client accs: [0.7506053447723389, 0.7968769669532776],alphas:tensor([7.5032e-01, 2.9302e-17, 2.4968e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:32,535 - utils - INFO - stage3_gradient_single_runtime: 0.006263017654418945
2023-09-28 23:32:32,540 - utils - INFO - 1, epoch: 1831, all client loss: [0.5097165703773499, 0.45555800199508667], all pred client disparities: [0.010520845651626587, 0.002026095986366272], all client disparities: [0.0032608509063720703, 0.0030545592308044434], all client accs: [0.7506053447723389, 0.7965347766876221],alphas:tensor([0.7504, 0.0000, 0.2496, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:32,769 - utils - INFO - stage3_gradient_single_runtime: 0.006358146667480469
2023-09-28 23:32:32,774 - utils - INFO - 1, epoch: 1832, all client loss: [0.5096950531005859, 0.45555827021598816], all pred client disparities: [0.010524272918701172, 0.0020227283239364624], all client disparities: [0.0032608509063720703, 0.0030545592308044434], all client accs: [0.7506053447723389, 0.7965347766876221],alphas:tensor([0.7505, 0.0000, 0.2495, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:33,004 - utils - INFO - stage3_gradient_single_runtime: 0.0062618255615234375
2023-09-28 23:32:33,009 - utils - INFO - 1, epoch: 1833, all client loss: [0.509673535823822, 0.45555856823921204], all pred client disparities: [0.01052767038345337, 0.0020193010568618774], all client disparities: [0.0032608509063720703, 0.002334088087081909], all client accs: [0.7506053447723389, 0.7962859272956848],alphas:tensor([7.5063e-01, 3.1097e-06, 2.4937e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:33,238 - utils - INFO - stage3_gradient_single_runtime: 0.006338357925415039
2023-09-28 23:32:33,242 - utils - INFO - 1, epoch: 1834, all client loss: [0.5096520781517029, 0.4555589258670807], all pred client disparities: [0.010531127452850342, 0.0020158886909484863], all client disparities: [0.0032608509063720703, 0.002334088087081909], all client accs: [0.7506053447723389, 0.7962548136711121],alphas:tensor([7.5073e-01, 4.1032e-05, 2.4923e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:33,471 - utils - INFO - stage3_gradient_single_runtime: 0.0063304901123046875
2023-09-28 23:32:33,476 - utils - INFO - 1, epoch: 1835, all client loss: [0.5096306204795837, 0.45555922389030457], all pred client disparities: [0.010534554719924927, 0.002012476325035095], all client disparities: [0.0032608509063720703, 0.002334088087081909], all client accs: [0.7506053447723389, 0.7962548136711121],alphas:tensor([7.5083e-01, 7.9504e-05, 2.4909e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:33,704 - utils - INFO - stage3_gradient_single_runtime: 0.006255626678466797
2023-09-28 23:32:33,709 - utils - INFO - 1, epoch: 1836, all client loss: [0.5096092820167542, 0.4555595815181732], all pred client disparities: [0.010538101196289062, 0.002008974552154541], all client disparities: [0.0032608509063720703, 0.002334088087081909], all client accs: [0.7506053447723389, 0.7962548136711121],alphas:tensor([7.5093e-01, 1.1801e-04, 2.4895e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:33,937 - utils - INFO - stage3_gradient_single_runtime: 0.006292819976806641
2023-09-28 23:32:33,942 - utils - INFO - 1, epoch: 1837, all client loss: [0.5095879435539246, 0.4555599093437195], all pred client disparities: [0.010541558265686035, 0.0020055025815963745], all client disparities: [0.0032608509063720703, 0.002407178282737732], all client accs: [0.7506053447723389, 0.7962859272956848],alphas:tensor([7.5102e-01, 1.5664e-04, 2.4882e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:34,170 - utils - INFO - stage3_gradient_single_runtime: 0.00628972053527832
2023-09-28 23:32:34,175 - utils - INFO - 1, epoch: 1838, all client loss: [0.509566605091095, 0.45556020736694336], all pred client disparities: [0.010545015335083008, 0.0020020008087158203], all client disparities: [0.0032608509063720703, 0.0017598122358322144], all client accs: [0.7506053447723389, 0.7963481545448303],alphas:tensor([7.5112e-01, 1.9567e-04, 2.4869e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:34,404 - utils - INFO - stage3_gradient_single_runtime: 0.006561279296875
2023-09-28 23:32:34,409 - utils - INFO - 1, epoch: 1839, all client loss: [0.5095453262329102, 0.455560564994812], all pred client disparities: [0.010548591613769531, 0.001998499035835266], all client disparities: [0.0032608509063720703, 0.0018329024314880371], all client accs: [0.7506053447723389, 0.7961614727973938],alphas:tensor([7.5121e-01, 2.3487e-04, 2.4855e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:34,488 - utils - INFO - valid: True, epoch: 1839, loss: [0.5758565068244934, 0.45464983582496643], accuracy: [0.7403315305709839, 0.798757791519165], mean_accuracy:0.7695446610450745,variance_accuracy:0.029213130474090576, disparity: [0.004545450210571289, 0.001856282353401184], mean_disparity:0.0032008662819862366,variance_disparity:0.0013445839285850525, pred_disparity: [0.004529297351837158, 0.0015099495649337769]
2023-09-28 23:32:34,621 - utils - INFO - global_valid: True, epoch: 1839,  global_loss: 0.4559973478317261, global_accuracy: 0.8274617088422982,  global_disparity:0.001256406307220459, global_pred_disparity: 0.004511132836341858,
2023-09-28 23:32:34,850 - utils - INFO - stage3_gradient_single_runtime: 0.006344318389892578
2023-09-28 23:32:34,855 - utils - INFO - 1, epoch: 1840, all client loss: [0.5095240473747253, 0.4555608630180359], all pred client disparities: [0.010552167892456055, 0.0019949227571487427], all client disparities: [0.0032608509063720703, 0.0018329024314880371], all client accs: [0.7506053447723389, 0.7961614727973938],alphas:tensor([7.5131e-01, 2.7392e-04, 2.4842e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:35,084 - utils - INFO - stage3_gradient_single_runtime: 0.006376028060913086
2023-09-28 23:32:35,091 - utils - INFO - 1, epoch: 1841, all client loss: [0.5095028281211853, 0.45556125044822693], all pred client disparities: [0.010555684566497803, 0.0019913464784622192], all client disparities: [0.0032608509063720703, 0.0018329024314880371], all client accs: [0.7506053447723389, 0.7960993051528931],alphas:tensor([7.5140e-01, 3.1341e-04, 2.4829e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:35,320 - utils - INFO - stage3_gradient_single_runtime: 0.006299257278442383
2023-09-28 23:32:35,325 - utils - INFO - 1, epoch: 1842, all client loss: [0.5094816088676453, 0.4555615484714508], all pred client disparities: [0.010559409856796265, 0.001987740397453308], all client disparities: [0.0032608509063720703, 0.0019059926271438599], all client accs: [0.7506053447723389, 0.795974850654602],alphas:tensor([7.5149e-01, 3.5320e-04, 2.4816e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:35,553 - utils - INFO - stage3_gradient_single_runtime: 0.006301164627075195
2023-09-28 23:32:35,558 - utils - INFO - 1, epoch: 1843, all client loss: [0.50946044921875, 0.45556190609931946], all pred client disparities: [0.0105629563331604, 0.001984149217605591], all client disparities: [0.0032608509063720703, 0.0019059926271438599], all client accs: [0.7506053447723389, 0.795974850654602],alphas:tensor([7.5158e-01, 3.9304e-04, 2.4803e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:35,785 - utils - INFO - stage3_gradient_single_runtime: 0.00629425048828125
2023-09-28 23:32:35,790 - utils - INFO - 1, epoch: 1844, all client loss: [0.5094393491744995, 0.45556220412254333], all pred client disparities: [0.010566622018814087, 0.001980498433113098], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7959126234054565],alphas:tensor([7.5167e-01, 4.3317e-04, 2.4790e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:36,018 - utils - INFO - stage3_gradient_single_runtime: 0.006326198577880859
2023-09-28 23:32:36,023 - utils - INFO - 1, epoch: 1845, all client loss: [0.509418249130249, 0.455562561750412], all pred client disparities: [0.010570257902145386, 0.001976802945137024], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7958193421363831],alphas:tensor([7.5175e-01, 4.7343e-04, 2.4777e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:36,253 - utils - INFO - stage3_gradient_single_runtime: 0.006363868713378906
2023-09-28 23:32:36,258 - utils - INFO - 1, epoch: 1846, all client loss: [0.5093972086906433, 0.45556291937828064], all pred client disparities: [0.010573983192443848, 0.0019731372594833374], all client disparities: [0.0032608509063720703, 0.0012586265802383423], all client accs: [0.7506053447723389, 0.7958193421363831],alphas:tensor([7.5184e-01, 5.1378e-04, 2.4765e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:36,486 - utils - INFO - stage3_gradient_single_runtime: 0.006296396255493164
2023-09-28 23:32:36,491 - utils - INFO - 1, epoch: 1847, all client loss: [0.5093761682510376, 0.4555632174015045], all pred client disparities: [0.01057770848274231, 0.001969441771507263], all client disparities: [0.0032608509063720703, 0.0012586265802383423], all client accs: [0.7506053447723389, 0.7957882285118103],alphas:tensor([7.5193e-01, 5.5430e-04, 2.4752e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:36,719 - utils - INFO - stage3_gradient_single_runtime: 0.006261348724365234
2023-09-28 23:32:36,724 - utils - INFO - 1, epoch: 1848, all client loss: [0.5093551874160767, 0.45556357502937317], all pred client disparities: [0.010581403970718384, 0.001965731382369995], all client disparities: [0.0032608509063720703, 0.0012586265802383423], all client accs: [0.7506053447723389, 0.7957882285118103],alphas:tensor([7.5201e-01, 5.9495e-04, 2.4739e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:36,953 - utils - INFO - stage3_gradient_single_runtime: 0.006318569183349609
2023-09-28 23:32:36,958 - utils - INFO - 1, epoch: 1849, all client loss: [0.5093342065811157, 0.45556390285491943], all pred client disparities: [0.010585218667984009, 0.0019619613885879517], all client disparities: [0.0032608509063720703, 0.0012586265802383423], all client accs: [0.7506053447723389, 0.7957882285118103],alphas:tensor([7.5209e-01, 6.3574e-04, 2.4727e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:37,187 - utils - INFO - stage3_gradient_single_runtime: 0.006318330764770508
2023-09-28 23:32:37,191 - utils - INFO - 1, epoch: 1850, all client loss: [0.5093132853507996, 0.4555642306804657], all pred client disparities: [0.010588973760604858, 0.001958206295967102], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7957571148872375],alphas:tensor([7.5218e-01, 6.7673e-04, 2.4715e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:37,420 - utils - INFO - stage3_gradient_single_runtime: 0.006293296813964844
2023-09-28 23:32:37,425 - utils - INFO - 1, epoch: 1851, all client loss: [0.5092923641204834, 0.45556458830833435], all pred client disparities: [0.010592728853225708, 0.0019544214010238647], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7957571148872375],alphas:tensor([7.5226e-01, 7.1791e-04, 2.4702e-01, 0.0000e+00, 0.0000e+00],
       device='cuda:0', dtype=torch.float64)
2023-09-28 23:32:37,654 - utils - INFO - stage3_gradient_single_runtime: 0.006295680999755859
2023-09-28 23:32:37,659 - utils - INFO - 1, epoch: 1852, all client loss: [0.509271502494812, 0.4555648863315582], all pred client disparities: [0.01059657335281372, 0.0019506216049194336], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7957571148872375],alphas:tensor([0.7523, 0.0008, 0.2469, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:37,886 - utils - INFO - stage3_gradient_single_runtime: 0.006295442581176758
2023-09-28 23:32:37,891 - utils - INFO - 1, epoch: 1853, all client loss: [0.5092507004737854, 0.4555652141571045], all pred client disparities: [0.010600388050079346, 0.001946777105331421], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7957260012626648],alphas:tensor([0.7524, 0.0008, 0.2468, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:38,117 - utils - INFO - stage3_gradient_single_runtime: 0.006304740905761719
2023-09-28 23:32:38,122 - utils - INFO - 1, epoch: 1854, all client loss: [0.5092298984527588, 0.45556554198265076], all pred client disparities: [0.010604262351989746, 0.0019429326057434082], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.795694887638092],alphas:tensor([0.7525, 0.0008, 0.2467, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:38,357 - utils - INFO - stage3_gradient_single_runtime: 0.0063402652740478516
2023-09-28 23:32:38,362 - utils - INFO - 1, epoch: 1855, all client loss: [0.509209156036377, 0.4555658996105194], all pred client disparities: [0.010608166456222534, 0.001939043402671814], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.795694887638092],alphas:tensor([0.7526, 0.0009, 0.2465, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:38,592 - utils - INFO - stage3_gradient_single_runtime: 0.006366252899169922
2023-09-28 23:32:38,597 - utils - INFO - 1, epoch: 1856, all client loss: [0.5091884136199951, 0.4555662274360657], all pred client disparities: [0.010612040758132935, 0.0019351840019226074], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.795694887638092],alphas:tensor([0.7527, 0.0009, 0.2464, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:38,826 - utils - INFO - stage3_gradient_single_runtime: 0.006270170211791992
2023-09-28 23:32:38,831 - utils - INFO - 1, epoch: 1857, all client loss: [0.5091677308082581, 0.4555665850639343], all pred client disparities: [0.01061597466468811, 0.0019312798976898193], all client disparities: [0.0032608509063720703, 0.0019059926271438599], all client accs: [0.7506053447723389, 0.7957882285118103],alphas:tensor([0.7527, 0.0010, 0.2463, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:39,066 - utils - INFO - stage3_gradient_single_runtime: 0.006295204162597656
2023-09-28 23:32:39,071 - utils - INFO - 1, epoch: 1858, all client loss: [0.5091471076011658, 0.4555668830871582], all pred client disparities: [0.01061999797821045, 0.0019273161888122559], all client disparities: [0.0032608509063720703, 0.0019059926271438599], all client accs: [0.7506053447723389, 0.7957882285118103],alphas:tensor([0.7528, 0.0010, 0.2462, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:39,301 - utils - INFO - stage3_gradient_single_runtime: 0.00625920295715332
2023-09-28 23:32:39,306 - utils - INFO - 1, epoch: 1859, all client loss: [0.5091264843940735, 0.45556721091270447], all pred client disparities: [0.010623931884765625, 0.001923397183418274], all client disparities: [0.0032608509063720703, 0.0019059926271438599], all client accs: [0.7506053447723389, 0.7957882285118103],alphas:tensor([0.7529, 0.0011, 0.2461, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:39,385 - utils - INFO - valid: True, epoch: 1859, loss: [0.5757596492767334, 0.454648494720459], accuracy: [0.7403315305709839, 0.7976397275924683], mean_accuracy:0.7689856290817261,variance_accuracy:0.028654098510742188, disparity: [0.004545450210571289, 0.0006858408451080322], mean_disparity:0.0026156455278396606,variance_disparity:0.0019298046827316284, pred_disparity: [0.004575282335281372, 0.001908332109451294]
2023-09-28 23:32:39,514 - utils - INFO - global_valid: True, epoch: 1859,  global_loss: 0.45599496364593506, global_accuracy: 0.8276066957435032,  global_disparity:0.0023743659257888794, global_pred_disparity: 0.004879280924797058,
2023-09-28 23:32:39,757 - utils - INFO - stage3_gradient_single_runtime: 0.006574153900146484
2023-09-28 23:32:39,762 - utils - INFO - 1, epoch: 1860, all client loss: [0.5091058611869812, 0.4555675685405731], all pred client disparities: [0.0106278657913208, 0.0019194483757019043], all client disparities: [0.0032608509063720703, 0.0019059926271438599], all client accs: [0.7506053447723389, 0.7957260012626648],alphas:tensor([0.7530, 0.0011, 0.2460, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:39,998 - utils - INFO - stage3_gradient_single_runtime: 0.0063207149505615234
2023-09-28 23:32:40,002 - utils - INFO - 1, epoch: 1861, all client loss: [0.5090852975845337, 0.455567866563797], all pred client disparities: [0.010631859302520752, 0.0019154548645019531], all client disparities: [0.0032608509063720703, 0.0019059926271438599], all client accs: [0.7506053447723389, 0.7957260012626648],alphas:tensor([0.7530, 0.0011, 0.2458, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:40,235 - utils - INFO - stage3_gradient_single_runtime: 0.006308078765869141
2023-09-28 23:32:40,240 - utils - INFO - 1, epoch: 1862, all client loss: [0.509064793586731, 0.45556822419166565], all pred client disparities: [0.010635912418365479, 0.0019114315509796143], all client disparities: [0.0032608509063720703, 0.0019059926271438599], all client accs: [0.7506053447723389, 0.7957260012626648],alphas:tensor([0.7531, 0.0012, 0.2457, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:40,473 - utils - INFO - stage3_gradient_single_runtime: 0.006280422210693359
2023-09-28 23:32:40,478 - utils - INFO - 1, epoch: 1863, all client loss: [0.5090442895889282, 0.4555685818195343], all pred client disparities: [0.010639935731887817, 0.0019073933362960815], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7532, 0.0012, 0.2456, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:40,710 - utils - INFO - stage3_gradient_single_runtime: 0.006322622299194336
2023-09-28 23:32:40,715 - utils - INFO - 1, epoch: 1864, all client loss: [0.5090238451957703, 0.45556890964508057], all pred client disparities: [0.010643959045410156, 0.0019033551216125488], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7532, 0.0013, 0.2455, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:40,949 - utils - INFO - stage3_gradient_single_runtime: 0.006243228912353516
2023-09-28 23:32:40,954 - utils - INFO - 1, epoch: 1865, all client loss: [0.5090034008026123, 0.45556920766830444], all pred client disparities: [0.010648071765899658, 0.0018993020057678223], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7533, 0.0013, 0.2454, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:41,189 - utils - INFO - stage3_gradient_single_runtime: 0.006326436996459961
2023-09-28 23:32:41,194 - utils - INFO - 1, epoch: 1866, all client loss: [0.5089830160140991, 0.4555695354938507], all pred client disparities: [0.01065218448638916, 0.001895219087600708], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7534, 0.0013, 0.2453, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:41,428 - utils - INFO - stage3_gradient_single_runtime: 0.006279706954956055
2023-09-28 23:32:41,433 - utils - INFO - 1, epoch: 1867, all client loss: [0.5089626312255859, 0.455569863319397], all pred client disparities: [0.010656267404556274, 0.0018911361694335938], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7534, 0.0014, 0.2452, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:41,667 - utils - INFO - stage3_gradient_single_runtime: 0.006255149841308594
2023-09-28 23:32:41,672 - utils - INFO - 1, epoch: 1868, all client loss: [0.5089423656463623, 0.45557019114494324], all pred client disparities: [0.010660439729690552, 0.001887008547782898], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7535, 0.0014, 0.2451, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:41,904 - utils - INFO - stage3_gradient_single_runtime: 0.006235361099243164
2023-09-28 23:32:41,909 - utils - INFO - 1, epoch: 1869, all client loss: [0.5089220404624939, 0.4555705487728119], all pred client disparities: [0.010664522647857666, 0.0018828660249710083], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7536, 0.0015, 0.2450, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:42,142 - utils - INFO - stage3_gradient_single_runtime: 0.006308078765869141
2023-09-28 23:32:42,146 - utils - INFO - 1, epoch: 1870, all client loss: [0.508901834487915, 0.45557087659835815], all pred client disparities: [0.010668754577636719, 0.001878693699836731], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7536, 0.0015, 0.2448, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:42,329 - utils - INFO - stage3_gradient_single_runtime: 0.006245613098144531
2023-09-28 23:32:42,334 - utils - INFO - 1, epoch: 1871, all client loss: [0.5088815093040466, 0.4555712044239044], all pred client disparities: [0.010672926902770996, 0.0018745213747024536], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.7954771518707275],alphas:tensor([0.7537, 0.0016, 0.2447, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:42,563 - utils - INFO - stage3_gradient_single_runtime: 0.006215572357177734
2023-09-28 23:32:42,567 - utils - INFO - 1, epoch: 1872, all client loss: [0.5088613629341125, 0.45557156205177307], all pred client disparities: [0.010677158832550049, 0.0018703490495681763], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.7955082654953003],alphas:tensor([0.7538, 0.0016, 0.2446, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:42,795 - utils - INFO - stage3_gradient_single_runtime: 0.00624847412109375
2023-09-28 23:32:42,800 - utils - INFO - 1, epoch: 1873, all client loss: [0.5088411569595337, 0.45557186007499695], all pred client disparities: [0.010681331157684326, 0.0018661320209503174], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.7955082654953003],alphas:tensor([0.7538, 0.0016, 0.2445, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:43,027 - utils - INFO - stage3_gradient_single_runtime: 0.006266593933105469
2023-09-28 23:32:43,032 - utils - INFO - 1, epoch: 1874, all client loss: [0.5088210701942444, 0.4555721879005432], all pred client disparities: [0.010685592889785767, 0.0018619149923324585], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.7955082654953003],alphas:tensor([0.7539, 0.0017, 0.2444, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:43,259 - utils - INFO - stage3_gradient_single_runtime: 0.006237506866455078
2023-09-28 23:32:43,264 - utils - INFO - 1, epoch: 1875, all client loss: [0.5088009834289551, 0.45557254552841187], all pred client disparities: [0.010689884424209595, 0.001857653260231018], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.7954460382461548],alphas:tensor([0.7540, 0.0017, 0.2443, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:43,491 - utils - INFO - stage3_gradient_single_runtime: 0.006236553192138672
2023-09-28 23:32:43,496 - utils - INFO - 1, epoch: 1876, all client loss: [0.5087808966636658, 0.45557287335395813], all pred client disparities: [0.010694146156311035, 0.0018533915281295776], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7540, 0.0018, 0.2442, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:43,725 - utils - INFO - stage3_gradient_single_runtime: 0.00626373291015625
2023-09-28 23:32:43,730 - utils - INFO - 1, epoch: 1877, all client loss: [0.5087608695030212, 0.4555732011795044], all pred client disparities: [0.010698497295379639, 0.0018490999937057495], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7541, 0.0018, 0.2441, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:43,958 - utils - INFO - stage3_gradient_single_runtime: 0.006258964538574219
2023-09-28 23:32:43,963 - utils - INFO - 1, epoch: 1878, all client loss: [0.5087408423423767, 0.45557352900505066], all pred client disparities: [0.010702759027481079, 0.0018447935581207275], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7541, 0.0019, 0.2440, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:44,189 - utils - INFO - stage3_gradient_single_runtime: 0.006285667419433594
2023-09-28 23:32:44,194 - utils - INFO - 1, epoch: 1879, all client loss: [0.5087209343910217, 0.4555738866329193], all pred client disparities: [0.010707110166549683, 0.0018404722213745117], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7542, 0.0019, 0.2439, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:44,320 - utils - INFO - valid: True, epoch: 1879, loss: [0.5756646394729614, 0.4546473026275635], accuracy: [0.7403315305709839, 0.7973291873931885], mean_accuracy:0.7688303589820862,variance_accuracy:0.028498828411102295, disparity: [0.004545450210571289, 0.0006858408451080322], mean_disparity:0.0026156455278396606,variance_disparity:0.0019298046827316284, pred_disparity: [0.004628807306289673, 0.0023075342178344727]
2023-09-28 23:32:44,397 - utils - INFO - global_valid: True, epoch: 1879,  global_loss: 0.45599275827407837, global_accuracy: 0.827814202234354,  global_disparity:0.0023743659257888794, global_pred_disparity: 0.005249485373497009,
2023-09-28 23:32:44,625 - utils - INFO - stage3_gradient_single_runtime: 0.006267547607421875
2023-09-28 23:32:44,630 - utils - INFO - 1, epoch: 1880, all client loss: [0.5087010264396667, 0.4555741846561432], all pred client disparities: [0.010711431503295898, 0.001836150884628296], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.795383870601654],alphas:tensor([0.7542, 0.0020, 0.2438, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:44,858 - utils - INFO - stage3_gradient_single_runtime: 0.0062580108642578125
2023-09-28 23:32:44,863 - utils - INFO - 1, epoch: 1881, all client loss: [0.5086811184883118, 0.45557451248168945], all pred client disparities: [0.010715782642364502, 0.0018317997455596924], all client disparities: [0.0032608509063720703, 0.0019790828227996826], all client accs: [0.7506053447723389, 0.795383870601654],alphas:tensor([0.7543, 0.0020, 0.2437, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:45,092 - utils - INFO - stage3_gradient_single_runtime: 0.0062983036041259766
2023-09-28 23:32:45,096 - utils - INFO - 1, epoch: 1882, all client loss: [0.5086612105369568, 0.4555748403072357], all pred client disparities: [0.01072019338607788, 0.001827433705329895], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7544, 0.0020, 0.2436, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:45,326 - utils - INFO - stage3_gradient_single_runtime: 0.006253242492675781
2023-09-28 23:32:45,330 - utils - INFO - 1, epoch: 1883, all client loss: [0.5086413621902466, 0.45557522773742676], all pred client disparities: [0.010724633932113647, 0.00182303786277771], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7544, 0.0021, 0.2435, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:45,563 - utils - INFO - stage3_gradient_single_runtime: 0.0069408416748046875
2023-09-28 23:32:45,568 - utils - INFO - 1, epoch: 1884, all client loss: [0.5086215734481812, 0.45557552576065063], all pred client disparities: [0.010729044675827026, 0.001818642020225525], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7545, 0.0021, 0.2434, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:45,813 - utils - INFO - stage3_gradient_single_runtime: 0.006624460220336914
2023-09-28 23:32:45,818 - utils - INFO - 1, epoch: 1885, all client loss: [0.5086018443107605, 0.4555758535861969], all pred client disparities: [0.01073351502418518, 0.001814231276512146], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7545, 0.0022, 0.2433, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:46,058 - utils - INFO - stage3_gradient_single_runtime: 0.0063037872314453125
2023-09-28 23:32:46,063 - utils - INFO - 1, epoch: 1886, all client loss: [0.5085821151733398, 0.45557618141174316], all pred client disparities: [0.01073792576789856, 0.0018097758293151855], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7546, 0.0022, 0.2432, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:46,291 - utils - INFO - stage3_gradient_single_runtime: 0.006252288818359375
2023-09-28 23:32:46,296 - utils - INFO - 1, epoch: 1887, all client loss: [0.5085623860359192, 0.4555765390396118], all pred client disparities: [0.010742425918579102, 0.001805335283279419], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7546, 0.0023, 0.2431, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:46,525 - utils - INFO - stage3_gradient_single_runtime: 0.0063021183013916016
2023-09-28 23:32:46,530 - utils - INFO - 1, epoch: 1888, all client loss: [0.5085427165031433, 0.4555768668651581], all pred client disparities: [0.01074683666229248, 0.0018008649349212646], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.7954460382461548],alphas:tensor([0.7547, 0.0023, 0.2430, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:46,758 - utils - INFO - stage3_gradient_single_runtime: 0.006300687789916992
2023-09-28 23:32:46,763 - utils - INFO - 1, epoch: 1889, all client loss: [0.5085231065750122, 0.45557716488838196], all pred client disparities: [0.010751396417617798, 0.0017963647842407227], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.7954460382461548],alphas:tensor([0.7547, 0.0023, 0.2429, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:46,991 - utils - INFO - stage3_gradient_single_runtime: 0.006241321563720703
2023-09-28 23:32:46,996 - utils - INFO - 1, epoch: 1890, all client loss: [0.5085034966468811, 0.4555775225162506], all pred client disparities: [0.010755926370620728, 0.0017918795347213745], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.7954460382461548],alphas:tensor([0.7548, 0.0024, 0.2428, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:47,221 - utils - INFO - stage3_gradient_single_runtime: 0.00627899169921875
2023-09-28 23:32:47,226 - utils - INFO - 1, epoch: 1891, all client loss: [0.50848388671875, 0.4555778503417969], all pred client disparities: [0.010760456323623657, 0.0017873495817184448], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7548, 0.0024, 0.2427, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:47,453 - utils - INFO - stage3_gradient_single_runtime: 0.0062618255615234375
2023-09-28 23:32:47,458 - utils - INFO - 1, epoch: 1892, all client loss: [0.5084643363952637, 0.45557817816734314], all pred client disparities: [0.010764986276626587, 0.0017827898263931274], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7506053447723389, 0.795414924621582],alphas:tensor([0.7549, 0.0025, 0.2426, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:47,686 - utils - INFO - stage3_gradient_single_runtime: 0.006281614303588867
2023-09-28 23:32:47,691 - utils - INFO - 1, epoch: 1893, all client loss: [0.5084449052810669, 0.4555785357952118], all pred client disparities: [0.010769516229629517, 0.001778244972229004], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7530266642570496, 0.795383870601654],alphas:tensor([0.7549, 0.0025, 0.2425, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:47,919 - utils - INFO - stage3_gradient_single_runtime: 0.006294727325439453
2023-09-28 23:32:47,924 - utils - INFO - 1, epoch: 1894, all client loss: [0.5084254145622253, 0.45557886362075806], all pred client disparities: [0.010774195194244385, 0.0017736852169036865], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7530266642570496, 0.795383870601654],alphas:tensor([0.7550, 0.0026, 0.2425, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:48,157 - utils - INFO - stage3_gradient_single_runtime: 0.006337642669677734
2023-09-28 23:32:48,162 - utils - INFO - 1, epoch: 1895, all client loss: [0.5084059238433838, 0.45557916164398193], all pred client disparities: [0.010778725147247314, 0.0017690956592559814], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7530266642570496, 0.795383870601654],alphas:tensor([0.7550, 0.0026, 0.2424, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:48,392 - utils - INFO - stage3_gradient_single_runtime: 0.006293773651123047
2023-09-28 23:32:48,397 - utils - INFO - 1, epoch: 1896, all client loss: [0.508386492729187, 0.4555795192718506], all pred client disparities: [0.010783374309539795, 0.0017645061016082764], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7530266642570496, 0.7953527569770813],alphas:tensor([0.7551, 0.0026, 0.2423, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:48,624 - utils - INFO - stage3_gradient_single_runtime: 0.006308317184448242
2023-09-28 23:32:48,628 - utils - INFO - 1, epoch: 1897, all client loss: [0.5083671808242798, 0.4555797874927521], all pred client disparities: [0.010788023471832275, 0.0017598867416381836], all client disparities: [0.0032608509063720703, 0.0020521730184555054], all client accs: [0.7530266642570496, 0.7953527569770813],alphas:tensor([0.7551, 0.0027, 0.2422, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:48,858 - utils - INFO - stage3_gradient_single_runtime: 0.0063173770904541016
2023-09-28 23:32:48,863 - utils - INFO - 1, epoch: 1898, all client loss: [0.5083478093147278, 0.4555801749229431], all pred client disparities: [0.010792553424835205, 0.001755252480506897], all client disparities: [0.0032608509063720703, 0.002125263214111328], all client accs: [0.7530266642570496, 0.795383870601654],alphas:tensor([0.7552, 0.0027, 0.2421, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:49,091 - utils - INFO - stage3_gradient_single_runtime: 0.006386280059814453
2023-09-28 23:32:49,096 - utils - INFO - 1, epoch: 1899, all client loss: [0.5083284974098206, 0.4555805027484894], all pred client disparities: [0.010797381401062012, 0.0017506033182144165], all client disparities: [0.0032608509063720703, 0.002125263214111328], all client accs: [0.7530266642570496, 0.795383870601654],alphas:tensor([0.7552, 0.0028, 0.2420, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:49,178 - utils - INFO - valid: True, epoch: 1899, loss: [0.5755730271339417, 0.4546462893486023], accuracy: [0.7403315305709839, 0.7973291873931885], mean_accuracy:0.7688303589820862,variance_accuracy:0.028498828411102295, disparity: [0.004545450210571289, 0.0006858408451080322], mean_disparity:0.0026156455278396606,variance_disparity:0.0019298046827316284, pred_disparity: [0.004688382148742676, 0.002704322338104248]
2023-09-28 23:32:49,303 - utils - INFO - global_valid: True, epoch: 1899,  global_loss: 0.45599067211151123, global_accuracy: 0.8279753104076716,  global_disparity:0.0023743659257888794, global_pred_disparity: 0.00561852753162384,
2023-09-28 23:32:49,531 - utils - INFO - stage3_gradient_single_runtime: 0.006247043609619141
2023-09-28 23:32:49,536 - utils - INFO - 1, epoch: 1900, all client loss: [0.5083092451095581, 0.45558083057403564], all pred client disparities: [0.010802030563354492, 0.0017459392547607422], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.7954460382461548],alphas:tensor([0.7553, 0.0028, 0.2419, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:49,763 - utils - INFO - stage3_gradient_single_runtime: 0.006276845932006836
2023-09-28 23:32:49,768 - utils - INFO - 1, epoch: 1901, all client loss: [0.5082899332046509, 0.4555811583995819], all pred client disparities: [0.010806679725646973, 0.0017412751913070679], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.7954460382461548],alphas:tensor([0.7553, 0.0029, 0.2418, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:49,999 - utils - INFO - stage3_gradient_single_runtime: 0.006274223327636719
2023-09-28 23:32:50,004 - utils - INFO - 1, epoch: 1902, all client loss: [0.5082707405090332, 0.4555814564228058], all pred client disparities: [0.01081135869026184, 0.0017365813255310059], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.7954460382461548],alphas:tensor([0.7554, 0.0029, 0.2417, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:50,233 - utils - INFO - stage3_gradient_single_runtime: 0.006343364715576172
2023-09-28 23:32:50,238 - utils - INFO - 1, epoch: 1903, all client loss: [0.5082514882087708, 0.4555818438529968], all pred client disparities: [0.010816127061843872, 0.0017318576574325562], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7554, 0.0029, 0.2416, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:50,472 - utils - INFO - stage3_gradient_single_runtime: 0.006628751754760742
2023-09-28 23:32:50,476 - utils - INFO - 1, epoch: 1904, all client loss: [0.5082323551177979, 0.4555821716785431], all pred client disparities: [0.010820865631103516, 0.0017271488904953003], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7555, 0.0030, 0.2416, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:50,712 - utils - INFO - stage3_gradient_single_runtime: 0.006365776062011719
2023-09-28 23:32:50,717 - utils - INFO - 1, epoch: 1905, all client loss: [0.508213222026825, 0.45558246970176697], all pred client disparities: [0.010825634002685547, 0.0017224103212356567], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7555, 0.0030, 0.2415, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:50,952 - utils - INFO - stage3_gradient_single_runtime: 0.006299018859863281
2023-09-28 23:32:50,957 - utils - INFO - 1, epoch: 1906, all client loss: [0.5081941485404968, 0.45558279752731323], all pred client disparities: [0.010830461978912354, 0.0017176717519760132], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7555, 0.0031, 0.2414, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:51,184 - utils - INFO - stage3_gradient_single_runtime: 0.006272077560424805
2023-09-28 23:32:51,189 - utils - INFO - 1, epoch: 1907, all client loss: [0.5081751346588135, 0.4555831551551819], all pred client disparities: [0.010835260152816772, 0.001712888479232788], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7556, 0.0031, 0.2413, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:51,418 - utils - INFO - stage3_gradient_single_runtime: 0.006276607513427734
2023-09-28 23:32:51,423 - utils - INFO - 1, epoch: 1908, all client loss: [0.5081560611724854, 0.45558348298072815], all pred client disparities: [0.010839998722076416, 0.0017081350088119507], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7556, 0.0032, 0.2412, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:51,655 - utils - INFO - stage3_gradient_single_runtime: 0.0063211917877197266
2023-09-28 23:32:51,660 - utils - INFO - 1, epoch: 1909, all client loss: [0.5081371068954468, 0.4555838108062744], all pred client disparities: [0.010844796895980835, 0.0017033219337463379], all client disparities: [0.0032608509063720703, 0.0014778971672058105], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7557, 0.0032, 0.2411, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:51,939 - utils - INFO - stage3_gradient_single_runtime: 0.0062868595123291016
2023-09-28 23:32:51,944 - utils - INFO - 1, epoch: 1910, all client loss: [0.5081181526184082, 0.4555841386318207], all pred client disparities: [0.01084965467453003, 0.001698508858680725], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.7954460382461548],alphas:tensor([0.7557, 0.0033, 0.2410, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:52,199 - utils - INFO - stage3_gradient_single_runtime: 0.006333589553833008
2023-09-28 23:32:52,203 - utils - INFO - 1, epoch: 1911, all client loss: [0.5080991983413696, 0.45558446645736694], all pred client disparities: [0.010854512453079224, 0.0016936808824539185], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7558, 0.0033, 0.2410, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:52,441 - utils - INFO - stage3_gradient_single_runtime: 0.006391286849975586
2023-09-28 23:32:52,446 - utils - INFO - 1, epoch: 1912, all client loss: [0.5080803036689758, 0.4555847942829132], all pred client disparities: [0.01085934042930603, 0.0016888529062271118], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7558, 0.0033, 0.2409, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:52,681 - utils - INFO - stage3_gradient_single_runtime: 0.006338357925415039
2023-09-28 23:32:52,686 - utils - INFO - 1, epoch: 1913, all client loss: [0.508061408996582, 0.45558515191078186], all pred client disparities: [0.010864228010177612, 0.0016839951276779175], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7558, 0.0034, 0.2408, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:52,914 - utils - INFO - stage3_gradient_single_runtime: 0.006261348724365234
2023-09-28 23:32:52,919 - utils - INFO - 1, epoch: 1914, all client loss: [0.508042573928833, 0.45558544993400574], all pred client disparities: [0.010869145393371582, 0.0016790926456451416], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.795414924621582],alphas:tensor([0.7559, 0.0034, 0.2407, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:53,154 - utils - INFO - stage3_gradient_single_runtime: 0.006339550018310547
2023-09-28 23:32:53,159 - utils - INFO - 1, epoch: 1915, all client loss: [0.5080237984657288, 0.455585777759552], all pred client disparities: [0.010874003171920776, 0.0016742348670959473], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.7953527569770813],alphas:tensor([0.7559, 0.0035, 0.2406, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:53,394 - utils - INFO - stage3_gradient_single_runtime: 0.009822845458984375
2023-09-28 23:32:53,400 - utils - INFO - 1, epoch: 1916, all client loss: [0.5080050230026245, 0.4555860757827759], all pred client disparities: [0.010878890752792358, 0.0016693472862243652], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.7953527569770813],alphas:tensor([0.7560, 0.0035, 0.2405, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:53,668 - utils - INFO - stage3_gradient_single_runtime: 0.006700277328491211
2023-09-28 23:32:53,673 - utils - INFO - 1, epoch: 1917, all client loss: [0.5079862475395203, 0.4555864632129669], all pred client disparities: [0.010883808135986328, 0.0016644448041915894], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.7953527569770813],alphas:tensor([0.7560, 0.0036, 0.2405, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:53,903 - utils - INFO - stage3_gradient_single_runtime: 0.006303548812866211
2023-09-28 23:32:53,908 - utils - INFO - 1, epoch: 1918, all client loss: [0.507967472076416, 0.4555867910385132], all pred client disparities: [0.010888814926147461, 0.0016595125198364258], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.7953216433525085],alphas:tensor([0.7560, 0.0036, 0.2404, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:54,137 - utils - INFO - stage3_gradient_single_runtime: 0.006315708160400391
2023-09-28 23:32:54,142 - utils - INFO - 1, epoch: 1919, all client loss: [0.5079488158226013, 0.45558711886405945], all pred client disparities: [0.010893791913986206, 0.0016545802354812622], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.7953216433525085],alphas:tensor([0.7561, 0.0036, 0.2403, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:54,220 - utils - INFO - valid: True, epoch: 1919, loss: [0.57548588514328, 0.45464542508125305], accuracy: [0.7403315305709839, 0.7972670793533325], mean_accuracy:0.7687993049621582,variance_accuracy:0.028467774391174316, disparity: [0.004545450210571289, 0.0006858408451080322], mean_disparity:0.0026156455278396606,variance_disparity:0.0019298046827316284, pred_disparity: [0.004752904176712036, 0.0030960440635681152]
2023-09-28 23:32:54,345 - utils - INFO - global_valid: True, epoch: 1919,  global_loss: 0.45598888397216797, global_accuracy: 0.8279779659220143,  global_disparity:0.0023743659257888794, global_pred_disparity: 0.0059838443994522095,
2023-09-28 23:32:54,576 - utils - INFO - stage3_gradient_single_runtime: 0.0064601898193359375
2023-09-28 23:32:54,580 - utils - INFO - 1, epoch: 1920, all client loss: [0.5079301595687866, 0.4555874168872833], all pred client disparities: [0.010898768901824951, 0.001649603247642517], all client disparities: [0.0032608509063720703, 0.0015509873628616333], all client accs: [0.7530266642570496, 0.7953216433525085],alphas:tensor([0.7561, 0.0037, 0.2402, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:54,815 - utils - INFO - stage3_gradient_single_runtime: 0.006323814392089844
2023-09-28 23:32:54,819 - utils - INFO - 1, epoch: 1921, all client loss: [0.5079115629196167, 0.4555877447128296], all pred client disparities: [0.010903745889663696, 0.0016446411609649658], all client disparities: [0.0032608509063720703, 0.0026264488697052], all client accs: [0.7530266642570496, 0.7960993051528931],alphas:tensor([0.7561, 0.0037, 0.2401, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:55,002 - utils - INFO - stage3_gradient_single_runtime: 0.0062596797943115234
2023-09-28 23:32:55,007 - utils - INFO - 1, epoch: 1922, all client loss: [0.5078929662704468, 0.45558810234069824], all pred client disparities: [0.010908812284469604, 0.0016396790742874146], all client disparities: [0.0032608509063720703, 0.0026264488697052], all client accs: [0.7530266642570496, 0.7960993051528931],alphas:tensor([0.7562, 0.0038, 0.2401, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:55,239 - utils - INFO - stage3_gradient_single_runtime: 0.006363630294799805
2023-09-28 23:32:55,244 - utils - INFO - 1, epoch: 1923, all client loss: [0.5078744292259216, 0.4555884301662445], all pred client disparities: [0.010913759469985962, 0.0016346722841262817], all client disparities: [0.0032608509063720703, 0.0026264488697052], all client accs: [0.7530266642570496, 0.7960681915283203],alphas:tensor([0.7562, 0.0038, 0.2400, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:55,481 - utils - INFO - stage3_gradient_single_runtime: 0.0063076019287109375
2023-09-28 23:32:55,486 - utils - INFO - 1, epoch: 1924, all client loss: [0.5078558921813965, 0.45558875799179077], all pred client disparities: [0.010918736457824707, 0.001629650592803955], all client disparities: [0.0032608509063720703, 0.002699539065361023], all client accs: [0.7530266642570496, 0.7960993051528931],alphas:tensor([0.7563, 0.0038, 0.2399, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:55,716 - utils - INFO - stage3_gradient_single_runtime: 0.006343364715576172
2023-09-28 23:32:55,721 - utils - INFO - 1, epoch: 1925, all client loss: [0.5078373551368713, 0.45558908581733704], all pred client disparities: [0.010923832654953003, 0.0016246587038040161], all client disparities: [0.0032608509063720703, 0.002699539065361023], all client accs: [0.7530266642570496, 0.7960993051528931],alphas:tensor([0.7563, 0.0039, 0.2398, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:55,952 - utils - INFO - stage3_gradient_single_runtime: 0.006310701370239258
2023-09-28 23:32:55,956 - utils - INFO - 1, epoch: 1926, all client loss: [0.5078189373016357, 0.4555894434452057], all pred client disparities: [0.010928899049758911, 0.0016196370124816895], all client disparities: [0.0032608509063720703, 0.002699539065361023], all client accs: [0.7530266642570496, 0.7960993051528931],alphas:tensor([0.7563, 0.0039, 0.2397, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:56,188 - utils - INFO - stage3_gradient_single_runtime: 0.0063397884368896484
2023-09-28 23:32:56,193 - utils - INFO - 1, epoch: 1927, all client loss: [0.5078004598617554, 0.45558977127075195], all pred client disparities: [0.010933905839920044, 0.0016145706176757812], all client disparities: [0.0032608509063720703, 0.002699539065361023], all client accs: [0.7530266642570496, 0.7960993051528931],alphas:tensor([0.7564, 0.0040, 0.2397, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:56,429 - utils - INFO - stage3_gradient_single_runtime: 0.006373405456542969
2023-09-28 23:32:56,434 - utils - INFO - 1, epoch: 1928, all client loss: [0.5077821016311646, 0.4555900990962982], all pred client disparities: [0.010938972234725952, 0.001609504222869873], all client disparities: [0.0032608509063720703, 0.002699539065361023], all client accs: [0.7530266642570496, 0.7960681915283203],alphas:tensor([0.7564, 0.0040, 0.2396, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:56,666 - utils - INFO - stage3_gradient_single_runtime: 0.006283998489379883
2023-09-28 23:32:56,671 - utils - INFO - 1, epoch: 1929, all client loss: [0.507763683795929, 0.4555903971195221], all pred client disparities: [0.010944068431854248, 0.0016044378280639648], all client disparities: [0.0032608509063720703, 0.002699539065361023], all client accs: [0.7530266642570496, 0.7960370779037476],alphas:tensor([0.7564, 0.0041, 0.2395, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:56,900 - utils - INFO - stage3_gradient_single_runtime: 0.006332874298095703
2023-09-28 23:32:56,905 - utils - INFO - 1, epoch: 1930, all client loss: [0.5077452659606934, 0.45559072494506836], all pred client disparities: [0.010949194431304932, 0.0015993714332580566], all client disparities: [0.0032608509063720703, 0.002699539065361023], all client accs: [0.7530266642570496, 0.7960370779037476],alphas:tensor([0.7565, 0.0041, 0.2394, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:57,137 - utils - INFO - stage3_gradient_single_runtime: 0.006336688995361328
2023-09-28 23:32:57,142 - utils - INFO - 1, epoch: 1931, all client loss: [0.5077269673347473, 0.455591082572937], all pred client disparities: [0.010954290628433228, 0.001594260334968567], all client disparities: [0.0032608509063720703, 0.002699539065361023], all client accs: [0.7530266642570496, 0.7960059642791748],alphas:tensor([0.7565, 0.0041, 0.2394, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:57,372 - utils - INFO - stage3_gradient_single_runtime: 0.006386995315551758
2023-09-28 23:32:57,377 - utils - INFO - 1, epoch: 1932, all client loss: [0.507708728313446, 0.4555914103984833], all pred client disparities: [0.010959446430206299, 0.0015891343355178833], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7959126234054565],alphas:tensor([0.7565, 0.0042, 0.2393, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:57,610 - utils - INFO - stage3_gradient_single_runtime: 0.0063359737396240234
2023-09-28 23:32:57,615 - utils - INFO - 1, epoch: 1933, all client loss: [0.5076904296875, 0.45559173822402954], all pred client disparities: [0.01096460223197937, 0.0015840381383895874], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7959126234054565],alphas:tensor([0.7566, 0.0042, 0.2392, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:57,845 - utils - INFO - stage3_gradient_single_runtime: 0.006280422210693359
2023-09-28 23:32:57,850 - utils - INFO - 1, epoch: 1934, all client loss: [0.5076722502708435, 0.4555920362472534], all pred client disparities: [0.010969787836074829, 0.00157889723777771], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958815693855286],alphas:tensor([0.7566, 0.0043, 0.2391, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:58,087 - utils - INFO - stage3_gradient_single_runtime: 0.00637364387512207
2023-09-28 23:32:58,092 - utils - INFO - 1, epoch: 1935, all client loss: [0.5076540112495422, 0.4555923640727997], all pred client disparities: [0.010974884033203125, 0.0015737563371658325], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7566, 0.0043, 0.2391, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:58,323 - utils - INFO - stage3_gradient_single_runtime: 0.006312131881713867
2023-09-28 23:32:58,328 - utils - INFO - 1, epoch: 1936, all client loss: [0.5076358318328857, 0.45559272170066833], all pred client disparities: [0.010980099439620972, 0.0015685856342315674], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7567, 0.0044, 0.2390, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:58,557 - utils - INFO - stage3_gradient_single_runtime: 0.0062961578369140625
2023-09-28 23:32:58,562 - utils - INFO - 1, epoch: 1937, all client loss: [0.507617712020874, 0.4555930495262146], all pred client disparities: [0.010985344648361206, 0.0015634149312973022], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958815693855286],alphas:tensor([0.7567, 0.0044, 0.2389, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:58,792 - utils - INFO - stage3_gradient_single_runtime: 0.00630950927734375
2023-09-28 23:32:58,797 - utils - INFO - 1, epoch: 1938, all client loss: [0.5075996518135071, 0.45559337735176086], all pred client disparities: [0.010990530252456665, 0.0015582442283630371], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7567, 0.0044, 0.2388, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:59,027 - utils - INFO - stage3_gradient_single_runtime: 0.006279945373535156
2023-09-28 23:32:59,032 - utils - INFO - 1, epoch: 1939, all client loss: [0.5075815320014954, 0.45559370517730713], all pred client disparities: [0.010995745658874512, 0.0015530437231063843], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958815693855286],alphas:tensor([0.7568, 0.0045, 0.2388, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:59,159 - utils - INFO - valid: True, epoch: 1939, loss: [0.5754040479660034, 0.4546448588371277], accuracy: [0.7403315305709839, 0.7972670793533325], mean_accuracy:0.7687993049621582,variance_accuracy:0.028467774391174316, disparity: [0.004545450210571289, 0.0013721436262130737], mean_disparity:0.0029587969183921814,variance_disparity:0.0015866532921791077, pred_disparity: [0.004821062088012695, 0.0034805983304977417]
2023-09-28 23:32:59,238 - utils - INFO - global_valid: True, epoch: 1939,  global_loss: 0.4559873938560486, global_accuracy: 0.828254787610081,  global_disparity:0.0043519288301467896, global_pred_disparity: 0.006343349814414978,
2023-09-28 23:32:59,480 - utils - INFO - stage3_gradient_single_runtime: 0.006980180740356445
2023-09-28 23:32:59,485 - utils - INFO - 1, epoch: 1940, all client loss: [0.5075635313987732, 0.4555940330028534], all pred client disparities: [0.011000990867614746, 0.0015478283166885376], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7568, 0.0045, 0.2387, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:59,717 - utils - INFO - stage3_gradient_single_runtime: 0.006291866302490234
2023-09-28 23:32:59,722 - utils - INFO - 1, epoch: 1941, all client loss: [0.5075454711914062, 0.45559439063072205], all pred client disparities: [0.01100623607635498, 0.001542612910270691], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7568, 0.0046, 0.2386, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:32:59,953 - utils - INFO - stage3_gradient_single_runtime: 0.006341218948364258
2023-09-28 23:32:59,957 - utils - INFO - 1, epoch: 1942, all client loss: [0.5075274705886841, 0.4555947184562683], all pred client disparities: [0.011011511087417603, 0.0015373677015304565], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7569, 0.0046, 0.2385, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:00,189 - utils - INFO - stage3_gradient_single_runtime: 0.006304264068603516
2023-09-28 23:33:00,194 - utils - INFO - 1, epoch: 1943, all client loss: [0.5075095891952515, 0.4555950164794922], all pred client disparities: [0.01101672649383545, 0.0015321075916290283], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7569, 0.0046, 0.2385, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:00,429 - utils - INFO - stage3_gradient_single_runtime: 0.0066912174224853516
2023-09-28 23:33:00,434 - utils - INFO - 1, epoch: 1944, all client loss: [0.5074915885925293, 0.45559531450271606], all pred client disparities: [0.011022001504898071, 0.0015268772840499878], all client disparities: [0.0032608509063720703, 0.0027726292610168457], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7569, 0.0047, 0.2384, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:00,688 - utils - INFO - stage3_gradient_single_runtime: 0.006280422210693359
2023-09-28 23:33:00,693 - utils - INFO - 1, epoch: 1945, all client loss: [0.5074737071990967, 0.4555957019329071], all pred client disparities: [0.011027306318283081, 0.0015215873718261719], all client disparities: [0.0032608509063720703, 0.0028457194566726685], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7570, 0.0047, 0.2383, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:00,925 - utils - INFO - stage3_gradient_single_runtime: 0.006261110305786133
2023-09-28 23:33:00,930 - utils - INFO - 1, epoch: 1946, all client loss: [0.5074558258056641, 0.455595999956131], all pred client disparities: [0.01103261113166809, 0.0015163421630859375], all client disparities: [0.0032608509063720703, 0.0028457194566726685], all client accs: [0.7530266642570496, 0.7958193421363831],alphas:tensor([0.7570, 0.0048, 0.2383, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:01,161 - utils - INFO - stage3_gradient_single_runtime: 0.006313323974609375
2023-09-28 23:33:01,167 - utils - INFO - 1, epoch: 1947, all client loss: [0.5074379444122314, 0.45559635758399963], all pred client disparities: [0.011037886142730713, 0.0015110522508621216], all client disparities: [0.0032608509063720703, 0.0028457194566726685], all client accs: [0.7530266642570496, 0.7957260012626648],alphas:tensor([0.7570, 0.0048, 0.2382, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:01,398 - utils - INFO - stage3_gradient_single_runtime: 0.0063190460205078125
2023-09-28 23:33:01,403 - utils - INFO - 1, epoch: 1948, all client loss: [0.5074201822280884, 0.4555966556072235], all pred client disparities: [0.011043250560760498, 0.001505732536315918], all client disparities: [0.0032608509063720703, 0.0028457194566726685], all client accs: [0.7530266642570496, 0.7957260012626648],alphas:tensor([0.7570, 0.0048, 0.2381, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:01,634 - utils - INFO - stage3_gradient_single_runtime: 0.0063152313232421875
2023-09-28 23:33:01,639 - utils - INFO - 1, epoch: 1949, all client loss: [0.5074023604393005, 0.45559701323509216], all pred client disparities: [0.011048585176467896, 0.0015003979206085205], all client disparities: [0.0032608509063720703, 0.0028457194566726685], all client accs: [0.7530266642570496, 0.7957260012626648],alphas:tensor([0.7571, 0.0049, 0.2380, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:01,868 - utils - INFO - stage3_gradient_single_runtime: 0.006300687789916992
2023-09-28 23:33:01,873 - utils - INFO - 1, epoch: 1950, all client loss: [0.5073846578598022, 0.4555973410606384], all pred client disparities: [0.011053889989852905, 0.001495078206062317], all client disparities: [0.0032608509063720703, 0.003284245729446411], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7571, 0.0049, 0.2380, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:02,102 - utils - INFO - stage3_gradient_single_runtime: 0.00626683235168457
2023-09-28 23:33:02,107 - utils - INFO - 1, epoch: 1951, all client loss: [0.507366955280304, 0.4555976390838623], all pred client disparities: [0.011059314012527466, 0.0014897435903549194], all client disparities: [0.0032608509063720703, 0.003357335925102234], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7571, 0.0050, 0.2379, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:02,338 - utils - INFO - stage3_gradient_single_runtime: 0.006277322769165039
2023-09-28 23:33:02,343 - utils - INFO - 1, epoch: 1952, all client loss: [0.5073491930961609, 0.45559799671173096], all pred client disparities: [0.011064618825912476, 0.0014843940734863281], all client disparities: [0.0032608509063720703, 0.003357335925102234], all client accs: [0.7530266642570496, 0.7958504557609558],alphas:tensor([0.7572, 0.0050, 0.2378, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:02,573 - utils - INFO - stage3_gradient_single_runtime: 0.0063054561614990234
2023-09-28 23:33:02,578 - utils - INFO - 1, epoch: 1953, all client loss: [0.5073315501213074, 0.4555983245372772], all pred client disparities: [0.011070013046264648, 0.0014790445566177368], all client disparities: [0.0032608509063720703, 0.0034304261207580566], all client accs: [0.7530266642570496, 0.7958815693855286],alphas:tensor([0.7572, 0.0050, 0.2378, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:02,809 - utils - INFO - stage3_gradient_single_runtime: 0.006259441375732422
2023-09-28 23:33:02,814 - utils - INFO - 1, epoch: 1954, all client loss: [0.5073139071464539, 0.4555986523628235], all pred client disparities: [0.011075466871261597, 0.0014736652374267578], all client disparities: [0.0032608509063720703, 0.0034304261207580566], all client accs: [0.7530266642570496, 0.7958815693855286],alphas:tensor([0.7572, 0.0051, 0.2377, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:03,044 - utils - INFO - stage3_gradient_single_runtime: 0.006317615509033203
2023-09-28 23:33:03,049 - utils - INFO - 1, epoch: 1955, all client loss: [0.5072963237762451, 0.45559898018836975], all pred client disparities: [0.011080890893936157, 0.001468271017074585], all client disparities: [0.0032608509063720703, 0.0036078840494155884], all client accs: [0.7530266642570496, 0.7967213988304138],alphas:tensor([0.7573, 0.0051, 0.2376, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:03,279 - utils - INFO - stage3_gradient_single_runtime: 0.006273746490478516
2023-09-28 23:33:03,284 - utils - INFO - 1, epoch: 1956, all client loss: [0.5072787404060364, 0.4555993378162384], all pred client disparities: [0.011086225509643555, 0.0014628767967224121], all client disparities: [0.0032608509063720703, 0.0036078840494155884], all client accs: [0.7530266642570496, 0.7967213988304138],alphas:tensor([0.7573, 0.0052, 0.2375, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:03,513 - utils - INFO - stage3_gradient_single_runtime: 0.006320476531982422
2023-09-28 23:33:03,518 - utils - INFO - 1, epoch: 1957, all client loss: [0.5072611570358276, 0.4555996358394623], all pred client disparities: [0.011091679334640503, 0.001457497477531433], all client disparities: [0.0032608509063720703, 0.0036078840494155884], all client accs: [0.7530266642570496, 0.7967213988304138],alphas:tensor([0.7573, 0.0052, 0.2375, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:03,747 - utils - INFO - stage3_gradient_single_runtime: 0.006277322769165039
2023-09-28 23:33:03,752 - utils - INFO - 1, epoch: 1958, all client loss: [0.5072436928749084, 0.45559996366500854], all pred client disparities: [0.011097192764282227, 0.0014520883560180664], all client disparities: [0.0032608509063720703, 0.0036078840494155884], all client accs: [0.7530266642570496, 0.7967213988304138],alphas:tensor([0.7573, 0.0052, 0.2374, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:03,984 - utils - INFO - stage3_gradient_single_runtime: 0.006662130355834961
2023-09-28 23:33:03,989 - utils - INFO - 1, epoch: 1959, all client loss: [0.5072261691093445, 0.4556002914905548], all pred client disparities: [0.011102557182312012, 0.001446649432182312], all client disparities: [0.0032608509063720703, 0.0036078840494155884], all client accs: [0.7530266642570496, 0.7967213988304138],alphas:tensor([0.7574, 0.0053, 0.2373, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:04,070 - utils - INFO - valid: True, epoch: 1959, loss: [0.5753282308578491, 0.4546446204185486], accuracy: [0.7403315305709839, 0.7974534034729004], mean_accuracy:0.7688924670219421,variance_accuracy:0.028560936450958252, disparity: [0.004545450210571289, 0.0007674098014831543], mean_disparity:0.0026564300060272217,variance_disparity:0.0018890202045440674, pred_disparity: [0.004891782999038696, 0.003856375813484192]
2023-09-28 23:33:04,195 - utils - INFO - global_valid: True, epoch: 1959,  global_loss: 0.45598626136779785, global_accuracy: 0.8282226287041028,  global_disparity:0.0037506818771362305, global_pred_disparity: 0.0066953301429748535,
2023-09-28 23:33:04,426 - utils - INFO - stage3_gradient_single_runtime: 0.006341457366943359
2023-09-28 23:33:04,431 - utils - INFO - 1, epoch: 1960, all client loss: [0.5072087049484253, 0.4556006193161011], all pred client disparities: [0.01110801100730896, 0.0014412403106689453], all client disparities: [0.0032608509063720703, 0.003680974245071411], all client accs: [0.7530266642570496, 0.7967213988304138],alphas:tensor([0.7574, 0.0053, 0.2373, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:04,663 - utils - INFO - stage3_gradient_single_runtime: 0.006316184997558594
2023-09-28 23:33:04,668 - utils - INFO - 1, epoch: 1961, all client loss: [0.5071913003921509, 0.45560094714164734], all pred client disparities: [0.011113494634628296, 0.001435786485671997], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7968769669532776],alphas:tensor([0.7574, 0.0054, 0.2372, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:04,901 - utils - INFO - stage3_gradient_single_runtime: 0.006343364715576172
2023-09-28 23:33:04,906 - utils - INFO - 1, epoch: 1962, all client loss: [0.5071738958358765, 0.4556012749671936], all pred client disparities: [0.01111900806427002, 0.0014303475618362427], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7968769669532776],alphas:tensor([0.7575, 0.0054, 0.2371, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:05,139 - utils - INFO - stage3_gradient_single_runtime: 0.006300687789916992
2023-09-28 23:33:05,144 - utils - INFO - 1, epoch: 1963, all client loss: [0.507156491279602, 0.45560160279273987], all pred client disparities: [0.011124491691589355, 0.0014248490333557129], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7968769669532776],alphas:tensor([0.7575, 0.0054, 0.2371, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:05,376 - utils - INFO - stage3_gradient_single_runtime: 0.006334066390991211
2023-09-28 23:33:05,381 - utils - INFO - 1, epoch: 1964, all client loss: [0.5071391463279724, 0.45560193061828613], all pred client disparities: [0.011129915714263916, 0.0014193952083587646], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7968769669532776],alphas:tensor([0.7575, 0.0055, 0.2370, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:05,611 - utils - INFO - stage3_gradient_single_runtime: 0.006270885467529297
2023-09-28 23:33:05,616 - utils - INFO - 1, epoch: 1965, all client loss: [0.5071218013763428, 0.4556022882461548], all pred client disparities: [0.011135458946228027, 0.001413881778717041], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7968769669532776],alphas:tensor([0.7576, 0.0055, 0.2369, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:05,847 - utils - INFO - stage3_gradient_single_runtime: 0.006257534027099609
2023-09-28 23:33:05,852 - utils - INFO - 1, epoch: 1966, all client loss: [0.5071044564247131, 0.45560261607170105], all pred client disparities: [0.011141031980514526, 0.001408398151397705], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7968769669532776],alphas:tensor([0.7576, 0.0056, 0.2369, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:06,086 - utils - INFO - stage3_gradient_single_runtime: 0.0063021183013916016
2023-09-28 23:33:06,091 - utils - INFO - 1, epoch: 1967, all client loss: [0.507087230682373, 0.4556029140949249], all pred client disparities: [0.01114654541015625, 0.0014028847217559814], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7576, 0.0056, 0.2368, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:06,326 - utils - INFO - stage3_gradient_single_runtime: 0.006326913833618164
2023-09-28 23:33:06,331 - utils - INFO - 1, epoch: 1968, all client loss: [0.507070004940033, 0.4556032419204712], all pred client disparities: [0.011152029037475586, 0.0013973861932754517], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7576, 0.0056, 0.2367, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:06,565 - utils - INFO - stage3_gradient_single_runtime: 0.006349325180053711
2023-09-28 23:33:06,570 - utils - INFO - 1, epoch: 1969, all client loss: [0.5070527791976929, 0.45560359954833984], all pred client disparities: [0.011157631874084473, 0.0013918280601501465], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7577, 0.0057, 0.2367, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:06,804 - utils - INFO - stage3_gradient_single_runtime: 0.006366252899169922
2023-09-28 23:33:06,809 - utils - INFO - 1, epoch: 1970, all client loss: [0.5070355534553528, 0.4556038975715637], all pred client disparities: [0.011163204908370972, 0.0013863146305084229], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7577, 0.0057, 0.2366, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:07,043 - utils - INFO - stage3_gradient_single_runtime: 0.006361722946166992
2023-09-28 23:33:07,048 - utils - INFO - 1, epoch: 1971, all client loss: [0.5070183873176575, 0.4556042551994324], all pred client disparities: [0.011168748140335083, 0.0013807564973831177], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7577, 0.0057, 0.2365, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:07,282 - utils - INFO - stage3_gradient_single_runtime: 0.0062563419342041016
2023-09-28 23:33:07,287 - utils - INFO - 1, epoch: 1972, all client loss: [0.5070012807846069, 0.45560455322265625], all pred client disparities: [0.01117435097694397, 0.0013751983642578125], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7578, 0.0058, 0.2365, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:07,469 - utils - INFO - stage3_gradient_single_runtime: 0.0063250064849853516
2023-09-28 23:33:07,474 - utils - INFO - 1, epoch: 1973, all client loss: [0.5069842338562012, 0.4556048810482025], all pred client disparities: [0.011179924011230469, 0.0013696551322937012], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7578, 0.0058, 0.2364, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:07,702 - utils - INFO - stage3_gradient_single_runtime: 0.0063114166259765625
2023-09-28 23:33:07,707 - utils - INFO - 1, epoch: 1974, all client loss: [0.5069671273231506, 0.45560523867607117], all pred client disparities: [0.011185556650161743, 0.0013640671968460083], all client disparities: [0.0032608509063720703, 0.003252863883972168], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7578, 0.0059, 0.2363, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:07,951 - utils - INFO - stage3_gradient_single_runtime: 0.00634312629699707
2023-09-28 23:33:07,956 - utils - INFO - 1, epoch: 1975, all client loss: [0.5069500207901001, 0.45560556650161743], all pred client disparities: [0.011191129684448242, 0.0013584941625595093], all client disparities: [0.0032608509063720703, 0.0033259540796279907], all client accs: [0.7530266642570496, 0.7965970039367676],alphas:tensor([0.7578, 0.0059, 0.2363, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:08,184 - utils - INFO - stage3_gradient_single_runtime: 0.006304025650024414
2023-09-28 23:33:08,189 - utils - INFO - 1, epoch: 1976, all client loss: [0.5069330334663391, 0.4556058645248413], all pred client disparities: [0.011196792125701904, 0.0013528913259506226], all client disparities: [0.0032608509063720703, 0.0033259540796279907], all client accs: [0.7530266642570496, 0.7965347766876221],alphas:tensor([0.7579, 0.0059, 0.2362, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:08,424 - utils - INFO - stage3_gradient_single_runtime: 0.006283760070800781
2023-09-28 23:33:08,428 - utils - INFO - 1, epoch: 1977, all client loss: [0.5069160461425781, 0.4556061923503876], all pred client disparities: [0.011202365159988403, 0.0013472586870193481], all client disparities: [0.0032608509063720703, 0.0033259540796279907], all client accs: [0.7530266642570496, 0.7965347766876221],alphas:tensor([0.7579, 0.0060, 0.2361, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:08,657 - utils - INFO - stage3_gradient_single_runtime: 0.006254673004150391
2023-09-28 23:33:08,661 - utils - INFO - 1, epoch: 1978, all client loss: [0.5068990588188171, 0.4556065499782562], all pred client disparities: [0.011208027601242065, 0.0013416558504104614], all client disparities: [0.0032608509063720703, 0.0033259540796279907], all client accs: [0.7530266642570496, 0.7965347766876221],alphas:tensor([0.7579, 0.0060, 0.2361, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:08,887 - utils - INFO - stage3_gradient_single_runtime: 0.006236553192138672
2023-09-28 23:33:08,892 - utils - INFO - 1, epoch: 1979, all client loss: [0.5068821310997009, 0.4556068778038025], all pred client disparities: [0.011213719844818115, 0.001336023211479187], all client disparities: [0.0032608509063720703, 0.0018850266933441162], all client accs: [0.7530266642570496, 0.7965658903121948],alphas:tensor([0.7579, 0.0061, 0.2360, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:09,024 - utils - INFO - valid: True, epoch: 1979, loss: [0.5752589106559753, 0.45464468002319336], accuracy: [0.7403315305709839, 0.7976397275924683], mean_accuracy:0.7689856290817261,variance_accuracy:0.028654098510742188, disparity: [0.004545450210571289, 0.00017569959163665771], mean_disparity:0.0023605749011039734,variance_disparity:0.0021848753094673157, pred_disparity: [0.004963994026184082, 0.004222050309181213]
2023-09-28 23:33:09,101 - utils - INFO - global_valid: True, epoch: 1979,  global_loss: 0.4559856057167053, global_accuracy: 0.8283030468786101,  global_disparity:0.0031776130199432373, global_pred_disparity: 0.007038533687591553,
2023-09-28 23:33:09,329 - utils - INFO - stage3_gradient_single_runtime: 0.006325960159301758
2023-09-28 23:33:09,334 - utils - INFO - 1, epoch: 1980, all client loss: [0.5068652033805847, 0.45560720562934875], all pred client disparities: [0.01121935248374939, 0.001330360770225525], all client disparities: [0.0032608509063720703, 0.0017388463020324707], all client accs: [0.7530266642570496, 0.7968769669532776],alphas:tensor([0.7580, 0.0061, 0.2359, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:09,567 - utils - INFO - stage3_gradient_single_runtime: 0.006247520446777344
2023-09-28 23:33:09,572 - utils - INFO - 1, epoch: 1981, all client loss: [0.5068483352661133, 0.455607533454895], all pred client disparities: [0.011225014925003052, 0.0013247579336166382], all client disparities: [0.0032608509063720703, 0.0017388463020324707], all client accs: [0.7530266642570496, 0.7966902852058411],alphas:tensor([0.7580, 0.0061, 0.2359, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:09,801 - utils - INFO - stage3_gradient_single_runtime: 0.006344318389892578
2023-09-28 23:33:09,806 - utils - INFO - 1, epoch: 1982, all client loss: [0.5068314671516418, 0.4556078612804413], all pred client disparities: [0.011230647563934326, 0.001319095492362976], all client disparities: [0.0032608509063720703, 0.0017388463020324707], all client accs: [0.7530266642570496, 0.7966902852058411],alphas:tensor([0.7580, 0.0062, 0.2358, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:10,034 - utils - INFO - stage3_gradient_single_runtime: 0.006262302398681641
2023-09-28 23:33:10,039 - utils - INFO - 1, epoch: 1983, all client loss: [0.5068146586418152, 0.45560815930366516], all pred client disparities: [0.011236369609832764, 0.0013134181499481201], all client disparities: [0.0032608509063720703, 0.0017388463020324707], all client accs: [0.7530266642570496, 0.7966902852058411],alphas:tensor([0.7581, 0.0062, 0.2357, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:10,272 - utils - INFO - stage3_gradient_single_runtime: 0.006395101547241211
2023-09-28 23:33:10,277 - utils - INFO - 1, epoch: 1984, all client loss: [0.5067978501319885, 0.4556085169315338], all pred client disparities: [0.011242091655731201, 0.0013077408075332642], all client disparities: [0.0032608509063720703, 0.0017388463020324707], all client accs: [0.7530266642570496, 0.7966591715812683],alphas:tensor([0.7581, 0.0062, 0.2357, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:10,505 - utils - INFO - stage3_gradient_single_runtime: 0.006270885467529297
2023-09-28 23:33:10,510 - utils - INFO - 1, epoch: 1985, all client loss: [0.5067810416221619, 0.4556088149547577], all pred client disparities: [0.011247754096984863, 0.001302093267440796], all client disparities: [0.0032608509063720703, 0.0017388463020324707], all client accs: [0.7530266642570496, 0.7966591715812683],alphas:tensor([0.7581, 0.0063, 0.2356, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:10,738 - utils - INFO - stage3_gradient_single_runtime: 0.0063266754150390625
2023-09-28 23:33:10,743 - utils - INFO - 1, epoch: 1986, all client loss: [0.50676429271698, 0.45560917258262634], all pred client disparities: [0.011253505945205688, 0.001296401023864746], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7530266642570496, 0.7962859272956848],alphas:tensor([0.7581, 0.0063, 0.2355, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:10,974 - utils - INFO - stage3_gradient_single_runtime: 0.00633549690246582
2023-09-28 23:33:10,979 - utils - INFO - 1, epoch: 1987, all client loss: [0.5067476034164429, 0.4556094706058502], all pred client disparities: [0.011259227991104126, 0.0012906938791275024], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7530266642570496, 0.7962548136711121],alphas:tensor([0.7582, 0.0063, 0.2355, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:11,205 - utils - INFO - stage3_gradient_single_runtime: 0.006268739700317383
2023-09-28 23:33:11,210 - utils - INFO - 1, epoch: 1988, all client loss: [0.506730854511261, 0.4556097984313965], all pred client disparities: [0.011264920234680176, 0.0012849867343902588], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7530266642570496, 0.7962548136711121],alphas:tensor([0.7582, 0.0064, 0.2354, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:11,437 - utils - INFO - stage3_gradient_single_runtime: 0.006287574768066406
2023-09-28 23:33:11,442 - utils - INFO - 1, epoch: 1989, all client loss: [0.5067141652107239, 0.45561015605926514], all pred client disparities: [0.011270642280578613, 0.001279294490814209], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7530266642570496, 0.7961925864219666],alphas:tensor([0.7582, 0.0064, 0.2354, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:11,670 - utils - INFO - stage3_gradient_single_runtime: 0.0062983036041259766
2023-09-28 23:33:11,675 - utils - INFO - 1, epoch: 1990, all client loss: [0.5066975355148315, 0.455610454082489], all pred client disparities: [0.011276453733444214, 0.0012735575437545776], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.7961925864219666],alphas:tensor([0.7114, 0.0195, 0.2233, 0.0000, 0.0458], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:11,902 - utils - INFO - stage3_gradient_single_runtime: 0.00628972053527832
2023-09-28 23:33:11,907 - utils - INFO - 1, epoch: 1991, all client loss: [0.5066810250282288, 0.4556143879890442], all pred client disparities: [0.011277109384536743, 0.0012811869382858276], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.795974850654602],alphas:tensor([0.7584, 0.0064, 0.2352, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:12,135 - utils - INFO - stage3_gradient_single_runtime: 0.006243228912353516
2023-09-28 23:33:12,140 - utils - INFO - 1, epoch: 1992, all client loss: [0.5066644549369812, 0.45561471581459045], all pred client disparities: [0.011282950639724731, 0.001275390386581421], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.795974850654602],alphas:tensor([0.7113, 0.0195, 0.2231, 0.0000, 0.0461], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:12,368 - utils - INFO - stage3_gradient_single_runtime: 0.006324291229248047
2023-09-28 23:33:12,373 - utils - INFO - 1, epoch: 1993, all client loss: [0.5066478848457336, 0.45561864972114563], all pred client disparities: [0.011283546686172485, 0.0012830793857574463], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.795974850654602],alphas:tensor([0.7113, 0.0195, 0.2229, 0.0000, 0.0463], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:12,600 - utils - INFO - stage3_gradient_single_runtime: 0.0062716007232666016
2023-09-28 23:33:12,604 - utils - INFO - 1, epoch: 1994, all client loss: [0.5066314339637756, 0.4556226134300232], all pred client disparities: [0.011284232139587402, 0.0012907683849334717], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.795974850654602],alphas:tensor([0.7589, 0.0062, 0.2350, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:12,833 - utils - INFO - stage3_gradient_single_runtime: 0.0063130855560302734
2023-09-28 23:33:12,838 - utils - INFO - 1, epoch: 1995, all client loss: [0.5066148638725281, 0.45562297105789185], all pred client disparities: [0.011290043592453003, 0.0012849122285842896], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.795974850654602],alphas:tensor([0.7113, 0.0195, 0.2227, 0.0000, 0.0466], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:13,071 - utils - INFO - stage3_gradient_single_runtime: 0.0063016414642333984
2023-09-28 23:33:13,076 - utils - INFO - 1, epoch: 1996, all client loss: [0.5065984725952148, 0.455626904964447], all pred client disparities: [0.011290758848190308, 0.0012926310300827026], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.795974850654602],alphas:tensor([0.7591, 0.0061, 0.2348, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:13,303 - utils - INFO - stage3_gradient_single_runtime: 0.006299495697021484
2023-09-28 23:33:13,308 - utils - INFO - 1, epoch: 1997, all client loss: [0.5065819621086121, 0.45562729239463806], all pred client disparities: [0.011296629905700684, 0.0012867748737335205], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.795974850654602],alphas:tensor([0.7113, 0.0195, 0.2224, 0.0000, 0.0468], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:13,536 - utils - INFO - stage3_gradient_single_runtime: 0.006292581558227539
2023-09-28 23:33:13,541 - utils - INFO - 1, epoch: 1998, all client loss: [0.5065655708312988, 0.4556312561035156], all pred client disparities: [0.0112973153591156, 0.0012944936752319336], all client disparities: [0.0032608509063720703, 0.0021042972803115845], all client accs: [0.7554479837417603, 0.795974850654602],alphas:tensor([0.7112, 0.0195, 0.2223, 0.0000, 0.0470], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:13,772 - utils - INFO - stage3_gradient_single_runtime: 0.00626826286315918
2023-09-28 23:33:13,777 - utils - INFO - 1, epoch: 1999, all client loss: [0.5065492391586304, 0.45563530921936035], all pred client disparities: [0.011298000812530518, 0.0013023018836975098], all client disparities: [0.0032608509063720703, 0.0021773874759674072], all client accs: [0.7554479837417603, 0.7960059642791748],alphas:tensor([0.7595, 0.0059, 0.2346, 0.0000, 0.0000], device='cuda:0',
       dtype=torch.float64)
2023-09-28 23:33:13,856 - utils - INFO - valid: True, epoch: 1999, loss: [0.5752200484275818, 0.45466896891593933], accuracy: [0.7403315305709839, 0.7967081069946289], mean_accuracy:0.7685198187828064,variance_accuracy:0.02818828821182251, disparity: [0.004545450210571289, 0.0026644617319107056], mean_disparity:0.0036049559712409973,variance_disparity:0.0009404942393302917, pred_disparity: [0.005025267601013184, 0.004516959190368652]
2023-09-28 23:33:13,979 - utils - INFO - global_valid: True, epoch: 1999,  global_loss: 0.456009179353714, global_accuracy: 0.8284590740284214,  global_disparity:0.005556806921958923, global_pred_disparity: 0.007312580943107605,
2023-09-28 23:33:13,979 - utils - INFO - stage3_runtime: 127.78934383392334
