NEW TEMPLATE
a photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  10.0
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 0.45815523076479 iterated over 56250 steps satisfies differential privacy with eps = 10 and delta = 8.333333333333334e-06.
Noise Scale:  0.45815523076479
**********
Num Epochs: 30
tensor(2.0566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.840583324432373
*************************
**** on testing set *****
Accuracy Rate: 0.8342651724815369
*************************
tensor(1.8066, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5371, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4639, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5869, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6748, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6191, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7520, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5957, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6602, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6660, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6240, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5859, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5859, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5205, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4697, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5605, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6816, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9035333395004272
*************************
**** on testing set *****
Accuracy Rate: 0.8963658213615417
*************************
tensor(1.4648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4346, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5547, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5430, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7207, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6191, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3125, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5371, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7812, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7637, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7021, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5107, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4717, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5410, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7715, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4951, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5117, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5195, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.912933349609375
*************************
**** on testing set *****
Accuracy Rate: 0.9059504866600037
*************************
tensor(1.6543, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5264, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4336, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3613, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4727, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4736, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4258, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4863, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3848, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5098, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3633, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4297, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4297, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6270, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5684, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4385, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5738.6270632743835
Accuracy Rate: 0.9093450307846069
------------------------------------
NEW TEMPLATE
a blurry photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  10.0
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 0.45815523076479 iterated over 56250 steps satisfies differential privacy with eps = 10 and delta = 8.333333333333334e-06.
Noise Scale:  0.45815523076479
**********
Num Epochs: 30
tensor(1.8867, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8408499956130981
*************************
**** on testing set *****
Accuracy Rate: 0.8344648480415344
*************************
tensor(1.5762, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6621, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8301, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6914, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7246, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6211, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4502, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4395, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7168, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6357, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5596, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6113, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6582, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4053, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6348, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4395, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6914, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9030666947364807
*************************
**** on testing set *****
Accuracy Rate: 0.8961661458015442
*************************
tensor(1.5762, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6797, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6787, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7930, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4492, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4639, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9004, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7383, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5664, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8809, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6074, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5527, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4492, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7090, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6387, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7109, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7871, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6211, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9139666557312012
*************************
**** on testing set *****
Accuracy Rate: 0.9055510759353638
*************************
tensor(1.5938, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5732, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3887, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5215, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4922, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5742, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5039, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4551, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5518, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7598, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5107, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5615, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5908, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4277, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3809, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5716.238059282303
Accuracy Rate: 0.9096445441246033
------------------------------------
NEW TEMPLATE
a black and white photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  10.0
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 0.45815523076479 iterated over 56250 steps satisfies differential privacy with eps = 10 and delta = 8.333333333333334e-06.
Noise Scale:  0.45815523076479
**********
Num Epochs: 30
tensor(1.9873, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8389500379562378
*************************
**** on testing set *****
Accuracy Rate: 0.8336661458015442
*************************
tensor(1.8730, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6855, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0156, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5156, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6523, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7910, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5215, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6758, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5586, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6357, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6426, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5781, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5342, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8438, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6250, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5781, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7109, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5898, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4287, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9016333222389221
*************************
**** on testing set *****
Accuracy Rate: 0.8949680328369141
*************************
tensor(1.4512, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3730, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4414, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5059, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6426, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8916, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5977, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6523, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5137, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4570, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4004, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5635, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5596, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7363, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6865, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4580, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6406, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3613, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3418, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9129166603088379
*************************
**** on testing set *****
Accuracy Rate: 0.9021565318107605
*************************
tensor(1.5723, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3379, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5527, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4922, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6406, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5605, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5410, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7422, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6035, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5645, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6562, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4492, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3379, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6650, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5748.593939781189
Accuracy Rate: 0.9100438952445984
------------------------------------
NEW TEMPLATE
a photo of the clothing item {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  10.0
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 0.45815523076479 iterated over 56250 steps satisfies differential privacy with eps = 10 and delta = 8.333333333333334e-06.
Noise Scale:  0.45815523076479
**********
Num Epochs: 30
tensor(1.9688, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8439000248908997
*************************
**** on testing set *****
Accuracy Rate: 0.8375598788261414
*************************
tensor(1.6172, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7793, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6514, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5332, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5967, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6318, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8086, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5488, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5156, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5879, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4375, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8242, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4102, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3086, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5488, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7178, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9034333229064941
*************************
**** on testing set *****
Accuracy Rate: 0.8917731642723083
*************************
tensor(1.4131, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5488, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4199, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6777, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5020, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4551, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4414, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8594, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5107, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6660, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4883, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5840, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5908, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3770, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4473, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8340, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9678, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4824, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9131667017936707
*************************
**** on testing set *****
Accuracy Rate: 0.9011581540107727
*************************
tensor(1.4844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6309, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3232, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7158, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5576, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4922, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7031, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4502, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7148, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6230, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7676, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4004, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6035, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5391, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5703, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6582, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6172, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5733.245717287064
Accuracy Rate: 0.9053514003753662
------------------------------------
