Namespace(dataset='mnist', workers=4, model='mlp_bn', batch_size=60000, n_samples=None, alpha=0, lr=0.01, start_epoch=0, max_epoch=200, stepsize=0, gamma=0.0, momentum=0.7, trial_num=mu_07, save_dir='/home/zhangxj/workspace/HBF/results', eval_freq=1, print_freq=50, gpu='MIG-4501b1c5-7ed4-50a6-aeaa-38d0ef742f13', use_cpu=False, seed=1, confusion_matrix=False)
Currently using GPU: MIG-4501b1c5-7ed4-50a6-aeaa-38d0ef742f13
Using GPU: NVIDIA A100 80GB PCIe MIG 3g.40gb
Creating dataset: mnist
Creating model: mlp_bn
momentum: 0.7, lr: 0.01, alpha: 0
==> Epoch 1/200
Train Loss 2.4656457901000977, Train Accuracy 8.411666870117188
==> Epoch 2/200
Train Loss 2.2410025596618652, Train Accuracy 20.108333587646484
==> Epoch 3/200
Train Loss 1.9331990480422974, Train Accuracy 38.165000915527344
==> Epoch 4/200
Train Loss 1.6600065231323242, Train Accuracy 49.823333740234375
==> Epoch 5/200
Train Loss 1.4580820798873901, Train Accuracy 57.48833465576172
==> Epoch 6/200
Train Loss 1.3122812509536743, Train Accuracy 62.95833206176758
==> Epoch 7/200
Train Loss 1.202108383178711, Train Accuracy 67.07499694824219
==> Epoch 8/200
Train Loss 1.1146012544631958, Train Accuracy 70.17333221435547
==> Epoch 9/200
Train Loss 1.0427502393722534, Train Accuracy 72.70333099365234
==> Epoch 10/200
Train Loss 0.9826957583427429, Train Accuracy 74.6933364868164
==> Epoch 11/200
Train Loss 0.9320307374000549, Train Accuracy 76.42166900634766
==> Epoch 12/200
Train Loss 0.8889937400817871, Train Accuracy 77.7300033569336
==> Epoch 13/200
Train Loss 0.852149248123169, Train Accuracy 78.89666748046875
==> Epoch 14/200
Train Loss 0.8202903270721436, Train Accuracy 79.8316650390625
==> Epoch 15/200
Train Loss 0.7924171686172485, Train Accuracy 80.64167022705078
==> Epoch 16/200
Train Loss 0.7677291631698608, Train Accuracy 81.36166381835938
==> Epoch 17/200
Train Loss 0.7456055879592896, Train Accuracy 81.97333526611328
==> Epoch 18/200
Train Loss 0.7255781292915344, Train Accuracy 82.50333404541016
==> Epoch 19/200
Train Loss 0.707297146320343, Train Accuracy 82.92500305175781
==> Epoch 20/200
Train Loss 0.690500020980835, Train Accuracy 83.30833435058594
==> Epoch 21/200
Train Loss 0.6749857068061829, Train Accuracy 83.71833038330078
==> Epoch 22/200
Train Loss 0.660595178604126, Train Accuracy 84.01667022705078
==> Epoch 23/200
Train Loss 0.6471983194351196, Train Accuracy 84.3116683959961
==> Epoch 24/200
Train Loss 0.6346858143806458, Train Accuracy 84.5616683959961
==> Epoch 25/200
Train Loss 0.6229639053344727, Train Accuracy 84.80166625976562
==> Epoch 26/200
Train Loss 0.6119518280029297, Train Accuracy 85.04499816894531
==> Epoch 27/200
Train Loss 0.6015793085098267, Train Accuracy 85.30833435058594
==> Epoch 28/200
Train Loss 0.5917853116989136, Train Accuracy 85.51166534423828
==> Epoch 29/200
Train Loss 0.5825165510177612, Train Accuracy 85.71333312988281
==> Epoch 30/200
Train Loss 0.5737268328666687, Train Accuracy 85.93000030517578
==> Epoch 31/200
Train Loss 0.5653753876686096, Train Accuracy 86.1050033569336
==> Epoch 32/200
Train Loss 0.5574265718460083, Train Accuracy 86.2699966430664
==> Epoch 33/200
Train Loss 0.5498486161231995, Train Accuracy 86.41000366210938
==> Epoch 34/200
Train Loss 0.5426133871078491, Train Accuracy 86.57166290283203
==> Epoch 35/200
Train Loss 0.5356956124305725, Train Accuracy 86.73999786376953
==> Epoch 36/200
Train Loss 0.5290727019309998, Train Accuracy 86.86833190917969
==> Epoch 37/200
Train Loss 0.5227242112159729, Train Accuracy 86.98500061035156
==> Epoch 38/200
Train Loss 0.5166314244270325, Train Accuracy 87.14167022705078
==> Epoch 39/200
Train Loss 0.5107776522636414, Train Accuracy 87.23500061035156
==> Epoch 40/200
Train Loss 0.5051474571228027, Train Accuracy 87.3566665649414
==> Epoch 41/200
Train Loss 0.499726802110672, Train Accuracy 87.46499633789062
==> Epoch 42/200
Train Loss 0.49450287222862244, Train Accuracy 87.55333709716797
==> Epoch 43/200
Train Loss 0.48946380615234375, Train Accuracy 87.65833282470703
==> Epoch 44/200
Train Loss 0.4845987856388092, Train Accuracy 87.73666381835938
==> Epoch 45/200
Train Loss 0.4798978567123413, Train Accuracy 87.83499908447266
==> Epoch 46/200
Train Loss 0.4753516912460327, Train Accuracy 87.92166900634766
==> Epoch 47/200
Train Loss 0.47095173597335815, Train Accuracy 88.02999877929688
==> Epoch 48/200
Train Loss 0.4666901230812073, Train Accuracy 88.12833404541016
==> Epoch 49/200
Train Loss 0.4625595510005951, Train Accuracy 88.22000122070312
==> Epoch 50/200
Train Loss 0.4585530161857605, Train Accuracy 88.30333709716797
==> Epoch 51/200
Train Loss 0.4546642601490021, Train Accuracy 88.38666534423828
==> Epoch 52/200
Train Loss 0.4508872926235199, Train Accuracy 88.45500183105469
==> Epoch 53/200
Train Loss 0.44721660017967224, Train Accuracy 88.52166748046875
==> Epoch 54/200
Train Loss 0.44364699721336365, Train Accuracy 88.6066665649414
==> Epoch 55/200
Train Loss 0.44017356634140015, Train Accuracy 88.69166564941406
==> Epoch 56/200
Train Loss 0.43679186701774597, Train Accuracy 88.76166534423828
==> Epoch 57/200
Train Loss 0.43349751830101013, Train Accuracy 88.82833099365234
==> Epoch 58/200
Train Loss 0.43028655648231506, Train Accuracy 88.90499877929688
==> Epoch 59/200
Train Loss 0.4271552264690399, Train Accuracy 88.98332977294922
==> Epoch 60/200
Train Loss 0.42410001158714294, Train Accuracy 89.0683364868164
==> Epoch 61/200
Train Loss 0.4211174547672272, Train Accuracy 89.12166595458984
==> Epoch 62/200
Train Loss 0.41820451617240906, Train Accuracy 89.19999694824219
==> Epoch 63/200
Train Loss 0.41535821557044983, Train Accuracy 89.27166748046875
==> Epoch 64/200
Train Loss 0.41257572174072266, Train Accuracy 89.31999969482422
==> Epoch 65/200
Train Loss 0.40985435247421265, Train Accuracy 89.38999938964844
==> Epoch 66/200
Train Loss 0.407191663980484, Train Accuracy 89.42833709716797
==> Epoch 67/200
Train Loss 0.40458524227142334, Train Accuracy 89.49666595458984
==> Epoch 68/200
Train Loss 0.40203288197517395, Train Accuracy 89.54833221435547
==> Epoch 69/200
Train Loss 0.39953234791755676, Train Accuracy 89.61333465576172
==> Epoch 70/200
Train Loss 0.39708176255226135, Train Accuracy 89.663330078125
==> Epoch 71/200
Train Loss 0.39467909932136536, Train Accuracy 89.72333526611328
==> Epoch 72/200
Train Loss 0.3923225998878479, Train Accuracy 89.77666473388672
==> Epoch 73/200
Train Loss 0.3900104761123657, Train Accuracy 89.8133316040039
==> Epoch 74/200
Train Loss 0.3877411484718323, Train Accuracy 89.87333679199219
==> Epoch 75/200
Train Loss 0.3855130076408386, Train Accuracy 89.93000030517578
==> Epoch 76/200
Train Loss 0.383324533700943, Train Accuracy 89.99166870117188
==> Epoch 77/200
Train Loss 0.3811744451522827, Train Accuracy 90.02833557128906
==> Epoch 78/200
Train Loss 0.37906119227409363, Train Accuracy 90.07333374023438
==> Epoch 79/200
Train Loss 0.3769836723804474, Train Accuracy 90.14167022705078
==> Epoch 80/200
Train Loss 0.3749404847621918, Train Accuracy 90.18000030517578
==> Epoch 81/200
Train Loss 0.3729305863380432, Train Accuracy 90.23833465576172
==> Epoch 82/200
Train Loss 0.3709527850151062, Train Accuracy 90.26166534423828
==> Epoch 83/200
Train Loss 0.36900609731674194, Train Accuracy 90.30999755859375
==> Epoch 84/200
Train Loss 0.3670893609523773, Train Accuracy 90.35333251953125
==> Epoch 85/200
Train Loss 0.3652017116546631, Train Accuracy 90.39167022705078
==> Epoch 86/200
Train Loss 0.36334219574928284, Train Accuracy 90.43000030517578
==> Epoch 87/200
Train Loss 0.36150988936424255, Train Accuracy 90.45999908447266
==> Epoch 88/200
Train Loss 0.3597039580345154, Train Accuracy 90.5
==> Epoch 89/200
Train Loss 0.35792356729507446, Train Accuracy 90.53666687011719
==> Epoch 90/200
Train Loss 0.3561679720878601, Train Accuracy 90.57499694824219
==> Epoch 91/200
Train Loss 0.35443639755249023, Train Accuracy 90.625
==> Epoch 92/200
Train Loss 0.35272809863090515, Train Accuracy 90.67333221435547
==> Epoch 93/200
Train Loss 0.35104238986968994, Train Accuracy 90.71499633789062
==> Epoch 94/200
Train Loss 0.34937864542007446, Train Accuracy 90.75833129882812
==> Epoch 95/200
Train Loss 0.3477362394332886, Train Accuracy 90.79833221435547
==> Epoch 96/200
Train Loss 0.34611451625823975, Train Accuracy 90.83833312988281
==> Epoch 97/200
Train Loss 0.3445129692554474, Train Accuracy 90.86666870117188
==> Epoch 98/200
Train Loss 0.3429309129714966, Train Accuracy 90.91000366210938
==> Epoch 99/200
Train Loss 0.3413679003715515, Train Accuracy 90.94000244140625
==> Epoch 100/200
Train Loss 0.3398234248161316, Train Accuracy 90.98332977294922
==> Epoch 101/200
Train Loss 0.33829694986343384, Train Accuracy 91.04000091552734
==> Epoch 102/200
Train Loss 0.33678802847862244, Train Accuracy 91.07166290283203
==> Epoch 103/200
Train Loss 0.3352961242198944, Train Accuracy 91.11333465576172
==> Epoch 104/200
Train Loss 0.3338209092617035, Train Accuracy 91.15499877929688
==> Epoch 105/200
Train Loss 0.3323619067668915, Train Accuracy 91.18000030517578
==> Epoch 106/200
Train Loss 0.33091872930526733, Train Accuracy 91.21833038330078
==> Epoch 107/200
Train Loss 0.3294909596443176, Train Accuracy 91.25166320800781
==> Epoch 108/200
Train Loss 0.3280782699584961, Train Accuracy 91.28166961669922
==> Epoch 109/200
Train Loss 0.3266802132129669, Train Accuracy 91.32333374023438
==> Epoch 110/200
Train Loss 0.3252965211868286, Train Accuracy 91.36499786376953
==> Epoch 111/200
Train Loss 0.3239268660545349, Train Accuracy 91.41666412353516
==> Epoch 112/200
Train Loss 0.3225708305835724, Train Accuracy 91.4566650390625
==> Epoch 113/200
Train Loss 0.3212282359600067, Train Accuracy 91.4816665649414
==> Epoch 114/200
Train Loss 0.31989869475364685, Train Accuracy 91.50833129882812
==> Epoch 115/200
Train Loss 0.3185819089412689, Train Accuracy 91.54000091552734
==> Epoch 116/200
Train Loss 0.3172776699066162, Train Accuracy 91.5683364868164
==> Epoch 117/200
Train Loss 0.31598567962646484, Train Accuracy 91.59666442871094
==> Epoch 118/200
Train Loss 0.3147056996822357, Train Accuracy 91.63500213623047
==> Epoch 119/200
Train Loss 0.31343743205070496, Train Accuracy 91.6683349609375
==> Epoch 120/200
Train Loss 0.3121807277202606, Train Accuracy 91.6933364868164
==> Epoch 121/200
Train Loss 0.31093528866767883, Train Accuracy 91.73332977294922
==> Epoch 122/200
Train Loss 0.3097008466720581, Train Accuracy 91.75499725341797
==> Epoch 123/200
Train Loss 0.3084773123264313, Train Accuracy 91.788330078125
==> Epoch 124/200
Train Loss 0.30726441740989685, Train Accuracy 91.8066635131836
==> Epoch 125/200
Train Loss 0.3060619831085205, Train Accuracy 91.83999633789062
==> Epoch 126/200
Train Loss 0.30486980080604553, Train Accuracy 91.87166595458984
==> Epoch 127/200
Train Loss 0.3036876916885376, Train Accuracy 91.89666748046875
==> Epoch 128/200
Train Loss 0.3025154173374176, Train Accuracy 91.93333435058594
==> Epoch 129/200
Train Loss 0.30135294795036316, Train Accuracy 91.961669921875
==> Epoch 130/200
Train Loss 0.30020004510879517, Train Accuracy 91.98999786376953
==> Epoch 131/200
Train Loss 0.2990565001964569, Train Accuracy 92.00333404541016
==> Epoch 132/200
Train Loss 0.29792219400405884, Train Accuracy 92.04000091552734
==> Epoch 133/200
Train Loss 0.2967970669269562, Train Accuracy 92.07833099365234
==> Epoch 134/200
Train Loss 0.29568085074424744, Train Accuracy 92.11666870117188
==> Epoch 135/200
Train Loss 0.29457351565361023, Train Accuracy 92.1449966430664
==> Epoch 136/200
Train Loss 0.29347479343414307, Train Accuracy 92.20166778564453
==> Epoch 137/200
Train Loss 0.29238462448120117, Train Accuracy 92.23666381835938
==> Epoch 138/200
Train Loss 0.29130294919013977, Train Accuracy 92.26667022705078
==> Epoch 139/200
Train Loss 0.2902296185493469, Train Accuracy 92.29000091552734
==> Epoch 140/200
Train Loss 0.2891644537448883, Train Accuracy 92.3133316040039
==> Epoch 141/200
Train Loss 0.28810736536979675, Train Accuracy 92.33499908447266
==> Epoch 142/200
Train Loss 0.2870582342147827, Train Accuracy 92.36666870117188
==> Epoch 143/200
Train Loss 0.2860170304775238, Train Accuracy 92.3933334350586
==> Epoch 144/200
Train Loss 0.2849836051464081, Train Accuracy 92.40833282470703
==> Epoch 145/200
Train Loss 0.2839577794075012, Train Accuracy 92.43000030517578
==> Epoch 146/200
Train Loss 0.2829395830631256, Train Accuracy 92.46833038330078
==> Epoch 147/200
Train Loss 0.2819288671016693, Train Accuracy 92.48666381835938
==> Epoch 148/200
Train Loss 0.28092554211616516, Train Accuracy 92.50666809082031
==> Epoch 149/200
Train Loss 0.2799295485019684, Train Accuracy 92.53333282470703
==> Epoch 150/200
Train Loss 0.27894076704978943, Train Accuracy 92.5616683959961
==> Epoch 151/200
Train Loss 0.27795910835266113, Train Accuracy 92.59166717529297
==> Epoch 152/200
Train Loss 0.2769845128059387, Train Accuracy 92.61333465576172
==> Epoch 153/200
Train Loss 0.2760169208049774, Train Accuracy 92.62666320800781
==> Epoch 154/200
Train Loss 0.2750561833381653, Train Accuracy 92.64833068847656
==> Epoch 155/200
Train Loss 0.2741023302078247, Train Accuracy 92.66999816894531
==> Epoch 156/200
Train Loss 0.27315521240234375, Train Accuracy 92.70166778564453
==> Epoch 157/200
Train Loss 0.2722148001194, Train Accuracy 92.73500061035156
==> Epoch 158/200
Train Loss 0.27128103375434875, Train Accuracy 92.75333404541016
==> Epoch 159/200
Train Loss 0.270353764295578, Train Accuracy 92.77333068847656
==> Epoch 160/200
Train Loss 0.26943302154541016, Train Accuracy 92.7933349609375
==> Epoch 161/200
Train Loss 0.26851868629455566, Train Accuracy 92.81999969482422
==> Epoch 162/200
Train Loss 0.26761072874069214, Train Accuracy 92.84166717529297
==> Epoch 163/200
Train Loss 0.2667090892791748, Train Accuracy 92.86333465576172
==> Epoch 164/200
Train Loss 0.2658136785030365, Train Accuracy 92.89666748046875
==> Epoch 165/200
Train Loss 0.26492443680763245, Train Accuracy 92.91500091552734
==> Epoch 166/200
Train Loss 0.26404130458831787, Train Accuracy 92.92166900634766
==> Epoch 167/200
Train Loss 0.2631642818450928, Train Accuracy 92.94833374023438
==> Epoch 168/200
Train Loss 0.2622932493686676, Train Accuracy 92.9749984741211
==> Epoch 169/200
Train Loss 0.26142823696136475, Train Accuracy 92.99333190917969
==> Epoch 170/200
Train Loss 0.2605690658092499, Train Accuracy 93.0183334350586
==> Epoch 171/200
Train Loss 0.2597157657146454, Train Accuracy 93.04666900634766
==> Epoch 172/200
Train Loss 0.2588682472705841, Train Accuracy 93.0683364868164
==> Epoch 173/200
Train Loss 0.2580265402793884, Train Accuracy 93.09833526611328
==> Epoch 174/200
Train Loss 0.257190465927124, Train Accuracy 93.11333465576172
==> Epoch 175/200
Train Loss 0.25636008381843567, Train Accuracy 93.12666320800781
==> Epoch 176/200
Train Loss 0.2555352747440338, Train Accuracy 93.15499877929688
==> Epoch 177/200
Train Loss 0.25471606850624084, Train Accuracy 93.18333435058594
==> Epoch 178/200
Train Loss 0.2539023458957672, Train Accuracy 93.19666290283203
==> Epoch 179/200
Train Loss 0.2530940771102905, Train Accuracy 93.22166442871094
==> Epoch 180/200
Train Loss 0.2522912323474884, Train Accuracy 93.23999786376953
==> Epoch 181/200
Train Loss 0.25149375200271606, Train Accuracy 93.26499938964844
==> Epoch 182/200
Train Loss 0.2507016062736511, Train Accuracy 93.28666687011719
==> Epoch 183/200
Train Loss 0.2499147206544876, Train Accuracy 93.31500244140625
==> Epoch 184/200
Train Loss 0.24913309514522552, Train Accuracy 93.336669921875
==> Epoch 185/200
Train Loss 0.2483566254377365, Train Accuracy 93.3499984741211
==> Epoch 186/200
Train Loss 0.24758535623550415, Train Accuracy 93.36333465576172
==> Epoch 187/200
Train Loss 0.2468191385269165, Train Accuracy 93.37833404541016
==> Epoch 188/200
Train Loss 0.24605800211429596, Train Accuracy 93.38666534423828
==> Epoch 189/200
Train Loss 0.24530188739299774, Train Accuracy 93.40833282470703
==> Epoch 190/200
Train Loss 0.24455074965953827, Train Accuracy 93.42500305175781
==> Epoch 191/200
Train Loss 0.24380457401275635, Train Accuracy 93.43333435058594
==> Epoch 192/200
Train Loss 0.243063285946846, Train Accuracy 93.461669921875
==> Epoch 193/200
Train Loss 0.24232682585716248, Train Accuracy 93.48666381835938
==> Epoch 194/200
Train Loss 0.24159517884254456, Train Accuracy 93.51000213623047
==> Epoch 195/200
Train Loss 0.24086833000183105, Train Accuracy 93.53166961669922
==> Epoch 196/200
Train Loss 0.2401462197303772, Train Accuracy 93.5633316040039
==> Epoch 197/200
Train Loss 0.2394288033246994, Train Accuracy 93.58000183105469
==> Epoch 198/200
Train Loss 0.2387160360813141, Train Accuracy 93.59833526611328
==> Epoch 199/200
Train Loss 0.23800787329673767, Train Accuracy 93.61333465576172
==> Epoch 200/200
Train Loss 0.23730431497097015, Train Accuracy 93.62999725341797
Finished. Total elapsed time (h:m:s): 0:29:58
