Namespace(dataset='mnist', workers=4, model='mlp_bn', batch_size=60000, n_samples=None, alpha=0, lr=0.01, start_epoch=0, max_epoch=200, stepsize=0, gamma=0.0, momentum=0.8, trial_num=mu_08, save_dir='/home/zhangxj/workspace/HBF/results', eval_freq=1, print_freq=50, gpu='0', use_cpu=False, seed=1, confusion_matrix=False)
Currently using GPU: 0
Using GPU: NVIDIA A100 80GB PCIe MIG 4g.40gb
Creating dataset: mnist
Creating model: mlp_bn
momentum: 0.8, lr: 0.01, alpha: 0
==> Epoch 1/200
Train Loss 2.4656457901000977, Train Accuracy 8.411666870117188
==> Epoch 2/200
Train Loss 2.2410025596618652, Train Accuracy 20.108333587646484
==> Epoch 3/200
Train Loss 1.916655421257019, Train Accuracy 38.948333740234375
==> Epoch 4/200
Train Loss 1.622234582901001, Train Accuracy 51.17166519165039
==> Epoch 5/200
Train Loss 1.4060994386672974, Train Accuracy 59.224998474121094
==> Epoch 6/200
Train Loss 1.2523366212844849, Train Accuracy 64.79000091552734
==> Epoch 7/200
Train Loss 1.1371538639068604, Train Accuracy 68.62666320800781
==> Epoch 8/200
Train Loss 1.0457372665405273, Train Accuracy 71.7300033569336
==> Epoch 9/200
Train Loss 0.9704179167747498, Train Accuracy 74.14167022705078
==> Epoch 10/200
Train Loss 0.9072725176811218, Train Accuracy 76.20999908447266
==> Epoch 11/200
Train Loss 0.854042112827301, Train Accuracy 77.87000274658203
==> Epoch 12/200
Train Loss 0.8091139197349548, Train Accuracy 79.2699966430664
==> Epoch 13/200
Train Loss 0.7711116075515747, Train Accuracy 80.39167022705078
==> Epoch 14/200
Train Loss 0.7387824654579163, Train Accuracy 81.33333587646484
==> Epoch 15/200
Train Loss 0.7110003232955933, Train Accuracy 82.14167022705078
==> Epoch 16/200
Train Loss 0.6867972612380981, Train Accuracy 82.79499816894531
==> Epoch 17/200
Train Loss 0.6653836965560913, Train Accuracy 83.34166717529297
==> Epoch 18/200
Train Loss 0.6461511850357056, Train Accuracy 83.8183364868164
==> Epoch 19/200
Train Loss 0.6286545395851135, Train Accuracy 84.21333312988281
==> Epoch 20/200
Train Loss 0.612583577632904, Train Accuracy 84.61833190917969
==> Epoch 21/200
Train Loss 0.5977294445037842, Train Accuracy 84.96666717529297
==> Epoch 22/200
Train Loss 0.5839512944221497, Train Accuracy 85.23833465576172
==> Epoch 23/200
Train Loss 0.5711491107940674, Train Accuracy 85.50333404541016
==> Epoch 24/200
Train Loss 0.5592430830001831, Train Accuracy 85.76333618164062
==> Epoch 25/200
Train Loss 0.5481614470481873, Train Accuracy 86.01667022705078
==> Epoch 26/200
Train Loss 0.537834107875824, Train Accuracy 86.25666809082031
==> Epoch 27/200
Train Loss 0.5281913876533508, Train Accuracy 86.4816665649414
==> Epoch 28/200
Train Loss 0.5191652178764343, Train Accuracy 86.65499877929688
==> Epoch 29/200
Train Loss 0.510691225528717, Train Accuracy 86.8183364868164
==> Epoch 30/200
Train Loss 0.5027106404304504, Train Accuracy 86.99500274658203
==> Epoch 31/200
Train Loss 0.49517160654067993, Train Accuracy 87.13833618164062
==> Epoch 32/200
Train Loss 0.4880296587944031, Train Accuracy 87.30166625976562
==> Epoch 33/200
Train Loss 0.4812469184398651, Train Accuracy 87.44166564941406
==> Epoch 34/200
Train Loss 0.474791556596756, Train Accuracy 87.57833099365234
==> Epoch 35/200
Train Loss 0.46863672137260437, Train Accuracy 87.72666931152344
==> Epoch 36/200
Train Loss 0.46275895833969116, Train Accuracy 87.87000274658203
==> Epoch 37/200
Train Loss 0.4571380317211151, Train Accuracy 87.98666381835938
==> Epoch 38/200
Train Loss 0.4517553746700287, Train Accuracy 88.12833404541016
==> Epoch 39/200
Train Loss 0.44659435749053955, Train Accuracy 88.22166442871094
==> Epoch 40/200
Train Loss 0.4416395425796509, Train Accuracy 88.32833099365234
==> Epoch 41/200
Train Loss 0.43687665462493896, Train Accuracy 88.413330078125
==> Epoch 42/200
Train Loss 0.43229252099990845, Train Accuracy 88.5
==> Epoch 43/200
Train Loss 0.42787501215934753, Train Accuracy 88.625
==> Epoch 44/200
Train Loss 0.42361295223236084, Train Accuracy 88.74500274658203
==> Epoch 45/200
Train Loss 0.4194962978363037, Train Accuracy 88.84333038330078
==> Epoch 46/200
Train Loss 0.4155156910419464, Train Accuracy 88.94499969482422
==> Epoch 47/200
Train Loss 0.4116627275943756, Train Accuracy 89.04833221435547
==> Epoch 48/200
Train Loss 0.4079297184944153, Train Accuracy 89.12666320800781
==> Epoch 49/200
Train Loss 0.40430954098701477, Train Accuracy 89.21666717529297
==> Epoch 50/200
Train Loss 0.40079575777053833, Train Accuracy 89.31500244140625
==> Epoch 51/200
Train Loss 0.39738234877586365, Train Accuracy 89.38666534423828
==> Epoch 52/200
Train Loss 0.3940638303756714, Train Accuracy 89.48666381835938
==> Epoch 53/200
Train Loss 0.3908349573612213, Train Accuracy 89.59500122070312
==> Epoch 54/200
Train Loss 0.38769105076789856, Train Accuracy 89.64833068847656
==> Epoch 55/200
Train Loss 0.3846276104450226, Train Accuracy 89.69499969482422
==> Epoch 56/200
Train Loss 0.3816404938697815, Train Accuracy 89.77833557128906
==> Epoch 57/200
Train Loss 0.3787258565425873, Train Accuracy 89.8550033569336
==> Epoch 58/200
Train Loss 0.37588000297546387, Train Accuracy 89.9383316040039
==> Epoch 59/200
Train Loss 0.37309959530830383, Train Accuracy 90.01499938964844
==> Epoch 60/200
Train Loss 0.3703816533088684, Train Accuracy 90.07166290283203
==> Epoch 61/200
Train Loss 0.3677229881286621, Train Accuracy 90.11833190917969
==> Epoch 62/200
Train Loss 0.3651209771633148, Train Accuracy 90.1866683959961
==> Epoch 63/200
Train Loss 0.36257290840148926, Train Accuracy 90.26499938964844
==> Epoch 64/200
Train Loss 0.3600764870643616, Train Accuracy 90.31666564941406
==> Epoch 65/200
Train Loss 0.3576294183731079, Train Accuracy 90.37333679199219
==> Epoch 66/200
Train Loss 0.3552294969558716, Train Accuracy 90.42666625976562
==> Epoch 67/200
Train Loss 0.35287466645240784, Train Accuracy 90.4816665649414
==> Epoch 68/200
Train Loss 0.35056307911872864, Train Accuracy 90.52333068847656
==> Epoch 69/200
Train Loss 0.3482929766178131, Train Accuracy 90.59166717529297
==> Epoch 70/200
Train Loss 0.346062570810318, Train Accuracy 90.6500015258789
==> Epoch 71/200
Train Loss 0.3438703417778015, Train Accuracy 90.711669921875
==> Epoch 72/200
Train Loss 0.3417147696018219, Train Accuracy 90.76667022705078
==> Epoch 73/200
Train Loss 0.33959439396858215, Train Accuracy 90.80999755859375
==> Epoch 74/200
Train Loss 0.3375079333782196, Train Accuracy 90.8499984741211
==> Epoch 75/200
Train Loss 0.3354541063308716, Train Accuracy 90.89833068847656
==> Epoch 76/200
Train Loss 0.3334316313266754, Train Accuracy 90.95999908447266
==> Epoch 77/200
Train Loss 0.33143946528434753, Train Accuracy 91.02833557128906
==> Epoch 78/200
Train Loss 0.3294764459133148, Train Accuracy 91.07666778564453
==> Epoch 79/200
Train Loss 0.32754167914390564, Train Accuracy 91.12999725341797
==> Epoch 80/200
Train Loss 0.32563406229019165, Train Accuracy 91.1816635131836
==> Epoch 81/200
Train Loss 0.3237527310848236, Train Accuracy 91.22333526611328
==> Epoch 82/200
Train Loss 0.32189688086509705, Train Accuracy 91.288330078125
==> Epoch 83/200
Train Loss 0.32006558775901794, Train Accuracy 91.33999633789062
==> Epoch 84/200
Train Loss 0.318258136510849, Train Accuracy 91.38500213623047
==> Epoch 85/200
Train Loss 0.3164738416671753, Train Accuracy 91.4383316040039
==> Epoch 86/200
Train Loss 0.3147118389606476, Train Accuracy 91.49333190917969
==> Epoch 87/200
Train Loss 0.31297165155410767, Train Accuracy 91.52666473388672
==> Epoch 88/200
Train Loss 0.31125253438949585, Train Accuracy 91.56999969482422
==> Epoch 89/200
Train Loss 0.3095538914203644, Train Accuracy 91.61666870117188
==> Epoch 90/200
Train Loss 0.30787524580955505, Train Accuracy 91.68000030517578
==> Epoch 91/200
Train Loss 0.30621597170829773, Train Accuracy 91.72333526611328
==> Epoch 92/200
Train Loss 0.3045755922794342, Train Accuracy 91.76667022705078
==> Epoch 93/200
Train Loss 0.3029536008834839, Train Accuracy 91.8116683959961
==> Epoch 94/200
Train Loss 0.30134961009025574, Train Accuracy 91.86166381835938
==> Epoch 95/200
Train Loss 0.2997630834579468, Train Accuracy 91.88833618164062
==> Epoch 96/200
Train Loss 0.29819369316101074, Train Accuracy 91.9183349609375
==> Epoch 97/200
Train Loss 0.2966409921646118, Train Accuracy 91.95500183105469
==> Epoch 98/200
Train Loss 0.29510462284088135, Train Accuracy 91.99500274658203
==> Epoch 99/200
Train Loss 0.29358425736427307, Train Accuracy 92.04833221435547
==> Epoch 100/200
Train Loss 0.29207953810691833, Train Accuracy 92.09666442871094
==> Epoch 101/200
Train Loss 0.2905901074409485, Train Accuracy 92.13333129882812
==> Epoch 102/200
Train Loss 0.2891157567501068, Train Accuracy 92.18000030517578
==> Epoch 103/200
Train Loss 0.2876560688018799, Train Accuracy 92.21833038330078
==> Epoch 104/200
Train Loss 0.2862108647823334, Train Accuracy 92.26499938964844
==> Epoch 105/200
Train Loss 0.2847798764705658, Train Accuracy 92.3066635131836
==> Epoch 106/200
Train Loss 0.28336283564567566, Train Accuracy 92.34333038330078
==> Epoch 107/200
Train Loss 0.2819594442844391, Train Accuracy 92.38999938964844
==> Epoch 108/200
Train Loss 0.28056958317756653, Train Accuracy 92.43000030517578
==> Epoch 109/200
Train Loss 0.27919304370880127, Train Accuracy 92.47000122070312
==> Epoch 110/200
Train Loss 0.27782949805259705, Train Accuracy 92.50333404541016
==> Epoch 111/200
Train Loss 0.2764787971973419, Train Accuracy 92.52666473388672
==> Epoch 112/200
Train Loss 0.27514082193374634, Train Accuracy 92.56999969482422
==> Epoch 113/200
Train Loss 0.2738153636455536, Train Accuracy 92.59833526611328
==> Epoch 114/200
Train Loss 0.27250218391418457, Train Accuracy 92.62999725341797
==> Epoch 115/200
Train Loss 0.2712011933326721, Train Accuracy 92.65833282470703
==> Epoch 116/200
Train Loss 0.2699122130870819, Train Accuracy 92.6933364868164
==> Epoch 117/200
Train Loss 0.268635094165802, Train Accuracy 92.73332977294922
==> Epoch 118/200
Train Loss 0.2673696279525757, Train Accuracy 92.75499725341797
==> Epoch 119/200
Train Loss 0.26611578464508057, Train Accuracy 92.7750015258789
==> Epoch 120/200
Train Loss 0.26487332582473755, Train Accuracy 92.8066635131836
==> Epoch 121/200
Train Loss 0.26364219188690186, Train Accuracy 92.836669921875
==> Epoch 122/200
Train Loss 0.26242223381996155, Train Accuracy 92.875
==> Epoch 123/200
Train Loss 0.2612132728099823, Train Accuracy 92.91000366210938
==> Epoch 124/200
Train Loss 0.2600153088569641, Train Accuracy 92.9316635131836
==> Epoch 125/200
Train Loss 0.2588281035423279, Train Accuracy 92.97000122070312
==> Epoch 126/200
Train Loss 0.25765156745910645, Train Accuracy 93.00499725341797
==> Epoch 127/200
Train Loss 0.2564856708049774, Train Accuracy 93.04499816894531
==> Epoch 128/200
Train Loss 0.2553302049636841, Train Accuracy 93.07499694824219
==> Epoch 129/200
Train Loss 0.25418511033058167, Train Accuracy 93.0999984741211
==> Epoch 130/200
Train Loss 0.253050297498703, Train Accuracy 93.12999725341797
==> Epoch 131/200
Train Loss 0.25192561745643616, Train Accuracy 93.15499877929688
==> Epoch 132/200
Train Loss 0.25081101059913635, Train Accuracy 93.18499755859375
==> Epoch 133/200
Train Loss 0.24970634281635284, Train Accuracy 93.22333526611328
==> Epoch 134/200
Train Loss 0.24861153960227966, Train Accuracy 93.24666595458984
==> Epoch 135/200
Train Loss 0.24752649664878845, Train Accuracy 93.27666473388672
==> Epoch 136/200
Train Loss 0.24645110964775085, Train Accuracy 93.30333709716797
==> Epoch 137/200
Train Loss 0.2453853189945221, Train Accuracy 93.32333374023438
==> Epoch 138/200
Train Loss 0.24432897567749023, Train Accuracy 93.36000061035156
==> Epoch 139/200
Train Loss 0.2432820051908493, Train Accuracy 93.38333129882812
==> Epoch 140/200
Train Loss 0.24224433302879333, Train Accuracy 93.40833282470703
==> Epoch 141/200
Train Loss 0.24121585488319397, Train Accuracy 93.44833374023438
==> Epoch 142/200
Train Loss 0.24019649624824524, Train Accuracy 93.4749984741211
==> Epoch 143/200
Train Loss 0.2391861379146576, Train Accuracy 93.50333404541016
==> Epoch 144/200
Train Loss 0.23818470537662506, Train Accuracy 93.54499816894531
==> Epoch 145/200
Train Loss 0.2371920943260193, Train Accuracy 93.56666564941406
==> Epoch 146/200
Train Loss 0.2362082153558731, Train Accuracy 93.58333587646484
==> Epoch 147/200
Train Loss 0.23523299396038055, Train Accuracy 93.60333251953125
==> Epoch 148/200
Train Loss 0.23426634073257446, Train Accuracy 93.625
==> Epoch 149/200
Train Loss 0.23330815136432648, Train Accuracy 93.64833068847656
==> Epoch 150/200
Train Loss 0.23235836625099182, Train Accuracy 93.67166900634766
==> Epoch 151/200
Train Loss 0.23141686618328094, Train Accuracy 93.69499969482422
==> Epoch 152/200
Train Loss 0.23048359155654907, Train Accuracy 93.73332977294922
==> Epoch 153/200
Train Loss 0.22955840826034546, Train Accuracy 93.75333404541016
==> Epoch 154/200
Train Loss 0.22864127159118652, Train Accuracy 93.77833557128906
==> Epoch 155/200
Train Loss 0.2277321070432663, Train Accuracy 93.80166625976562
==> Epoch 156/200
Train Loss 0.22683078050613403, Train Accuracy 93.81666564941406
==> Epoch 157/200
Train Loss 0.22593723237514496, Train Accuracy 93.84666442871094
==> Epoch 158/200
Train Loss 0.2250513732433319, Train Accuracy 93.88833618164062
==> Epoch 159/200
Train Loss 0.2241731435060501, Train Accuracy 93.90666961669922
==> Epoch 160/200
Train Loss 0.22330240905284882, Train Accuracy 93.93000030517578
==> Epoch 161/200
Train Loss 0.22243912518024445, Train Accuracy 93.95333099365234
==> Epoch 162/200
Train Loss 0.22158317267894745, Train Accuracy 93.96833038330078
==> Epoch 163/200
Train Loss 0.22073450684547424, Train Accuracy 93.98500061035156
==> Epoch 164/200
Train Loss 0.2198929786682129, Train Accuracy 93.99833679199219
==> Epoch 165/200
Train Loss 0.21905860304832458, Train Accuracy 94.0199966430664
==> Epoch 166/200
Train Loss 0.2182312309741974, Train Accuracy 94.05333709716797
==> Epoch 167/200
Train Loss 0.21741080284118652, Train Accuracy 94.08000183105469
==> Epoch 168/200
Train Loss 0.21659722924232483, Train Accuracy 94.10832977294922
==> Epoch 169/200
Train Loss 0.21579043567180634, Train Accuracy 94.125
==> Epoch 170/200
Train Loss 0.2149903029203415, Train Accuracy 94.1449966430664
==> Epoch 171/200
Train Loss 0.2141968160867691, Train Accuracy 94.17500305175781
==> Epoch 172/200
Train Loss 0.21340982615947723, Train Accuracy 94.19666290283203
==> Epoch 173/200
Train Loss 0.21262934803962708, Train Accuracy 94.22833251953125
==> Epoch 174/200
Train Loss 0.2118552178144455, Train Accuracy 94.25
==> Epoch 175/200
Train Loss 0.2110874205827713, Train Accuracy 94.26499938964844
==> Epoch 176/200
Train Loss 0.21032580733299255, Train Accuracy 94.27666473388672
==> Epoch 177/200
Train Loss 0.20957037806510925, Train Accuracy 94.30000305175781
==> Epoch 178/200
Train Loss 0.20882102847099304, Train Accuracy 94.31500244140625
==> Epoch 179/200
Train Loss 0.20807763934135437, Train Accuracy 94.33000183105469
==> Epoch 180/200
Train Loss 0.20734021067619324, Train Accuracy 94.35832977294922
==> Epoch 181/200
Train Loss 0.2066086381673813, Train Accuracy 94.375
==> Epoch 182/200
Train Loss 0.20588284730911255, Train Accuracy 94.40166473388672
==> Epoch 183/200
Train Loss 0.20516276359558105, Train Accuracy 94.43499755859375
==> Epoch 184/200
Train Loss 0.20444831252098083, Train Accuracy 94.44999694824219
==> Epoch 185/200
Train Loss 0.2037394493818283, Train Accuracy 94.4566650390625
==> Epoch 186/200
Train Loss 0.2030360847711563, Train Accuracy 94.47666931152344
==> Epoch 187/200
Train Loss 0.20233814418315887, Train Accuracy 94.5
==> Epoch 188/200
Train Loss 0.20164556801319122, Train Accuracy 94.53500366210938
==> Epoch 189/200
Train Loss 0.20095831155776978, Train Accuracy 94.55000305175781
==> Epoch 190/200
Train Loss 0.20027628540992737, Train Accuracy 94.57166290283203
==> Epoch 191/200
Train Loss 0.19959942996501923, Train Accuracy 94.57666778564453
==> Epoch 192/200
Train Loss 0.19892767071723938, Train Accuracy 94.59666442871094
==> Epoch 193/200
Train Loss 0.19826096296310425, Train Accuracy 94.61333465576172
==> Epoch 194/200
Train Loss 0.19759924709796906, Train Accuracy 94.6449966430664
==> Epoch 195/200
Train Loss 0.19694243371486664, Train Accuracy 94.65666961669922
==> Epoch 196/200
Train Loss 0.19629047811031342, Train Accuracy 94.67833709716797
==> Epoch 197/200
Train Loss 0.1956433206796646, Train Accuracy 94.69666290283203
==> Epoch 198/200
Train Loss 0.19500091671943665, Train Accuracy 94.71833038330078
==> Epoch 199/200
Train Loss 0.19436317682266235, Train Accuracy 94.73666381835938
==> Epoch 200/200
Train Loss 0.19373010098934174, Train Accuracy 94.74833679199219
Finished. Total elapsed time (h:m:s): 0:30:27
