NEW TEMPLATE
a photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  3.0
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 0.6202129462792134 iterated over 56250 steps satisfies differential privacy with eps = 3 and delta = 8.333333333333334e-06.
Noise Scale:  0.6202129462792134
**********
Num Epochs: 30
tensor(1.9736, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8463833332061768
*************************
**** on testing set *****
Accuracy Rate: 0.8404552340507507
*************************
tensor(1.9609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7266, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4785, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5547, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5918, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7070, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7676, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7715, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5254, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5527, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7998, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7031, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5352, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7471, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4805, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6475, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8965, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8623, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6934, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9038000106811523
*************************
**** on testing set *****
Accuracy Rate: 0.897164523601532
*************************
tensor(1.4668, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4297, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4424, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7363, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8340, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4678, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4453, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5801, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7285, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4512, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9688, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5674, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3428, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5791, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6660, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4238, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6943, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5781, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6895, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9134333729743958
*************************
**** on testing set *****
Accuracy Rate: 0.9043530225753784
*************************
tensor(1.4766, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5918, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4434, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6973, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3711, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6621, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3984, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5703, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3613, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4512, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5762, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4238, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4121, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5664, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4189, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5753.470732927322
Accuracy Rate: 0.9091453552246094
------------------------------------
NEW TEMPLATE
a blurry photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  3.0
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 0.6202129462792134 iterated over 56250 steps satisfies differential privacy with eps = 3 and delta = 8.333333333333334e-06.
Noise Scale:  0.6202129462792134
**********
Num Epochs: 30
tensor(1.7637, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8460167050361633
*************************
**** on testing set *****
Accuracy Rate: 0.8393570184707642
*************************
tensor(1.6924, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8848, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7715, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5088, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5732, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5732, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6748, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6260, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6934, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6748, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6543, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5078, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6230, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4756, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5508, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6895, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3574, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7500, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9039333462715149
*************************
**** on testing set *****
Accuracy Rate: 0.8965654969215393
*************************
tensor(1.6387, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9062, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5840, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6279, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4238, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5723, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5059, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6904, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5508, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7051, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4219, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3809, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4561, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7812, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6621, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6191, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4570, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9140000343322754
*************************
**** on testing set *****
Accuracy Rate: 0.9050518870353699
*************************
tensor(1.7646, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6172, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6768, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6699, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4902, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6816, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4385, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4160, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5029, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6582, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5498, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3945, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5371, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8223, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5889, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5957, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6221, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5763.776942491531
Accuracy Rate: 0.9081469774246216
------------------------------------
NEW TEMPLATE
a black and white photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  3.0
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 0.6202129462792134 iterated over 56250 steps satisfies differential privacy with eps = 3 and delta = 8.333333333333334e-06.
Noise Scale:  0.6202129462792134
**********
Num Epochs: 30
tensor(2.2461, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8458833694458008
*************************
**** on testing set *****
Accuracy Rate: 0.8406549096107483
*************************
tensor(1.8594, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7715, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7500, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7090, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7012, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6660, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5654, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6758, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5098, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6953, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5303, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5605, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4980, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6064, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7363, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6025, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9036999940872192
*************************
**** on testing set *****
Accuracy Rate: 0.8966653347015381
*************************
tensor(1.5215, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7676, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5205, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3867, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7109, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4580, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4199, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8047, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5576, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3828, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5859, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4775, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4980, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5801, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5508, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6992, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9146833419799805
*************************
**** on testing set *****
Accuracy Rate: 0.9033546447753906
*************************
tensor(1.8184, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4727, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4326, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6602, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4395, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6191, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4551, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4033, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4160, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6660, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5371, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4551, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5703, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7148, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5449, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5816.577594280243
Accuracy Rate: 0.9073482155799866
------------------------------------
NEW TEMPLATE
a photo of the clothing item {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  3.0
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 0.6202129462792134 iterated over 56250 steps satisfies differential privacy with eps = 3 and delta = 8.333333333333334e-06.
Noise Scale:  0.6202129462792134
**********
Num Epochs: 30
tensor(1.8789, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8496833443641663
*************************
**** on testing set *****
Accuracy Rate: 0.8432508111000061
*************************
tensor(1.6309, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6641, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6250, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6074, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4912, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6914, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7910, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7676, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5205, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7822, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5137, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7969, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6270, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4629, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5957, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6016, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7979, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4502, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5352, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9038666486740112
*************************
**** on testing set *****
Accuracy Rate: 0.8972643613815308
*************************
tensor(1.5059, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8174, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4902, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6631, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3184, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5166, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5098, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4238, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3613, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6113, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4473, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4971, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6504, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5547, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5254, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5098, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6348, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4570, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9144333600997925
*************************
**** on testing set *****
Accuracy Rate: 0.9049520492553711
*************************
tensor(1.5918, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4932, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4473, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4746, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4746, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4219, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3242, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5723, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6045, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7871, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3213, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6035, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5234, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5000, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5816.365476369858
Accuracy Rate: 0.9065495133399963
------------------------------------
