NEW TEMPLATE
a photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  0.5
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 1.2805631372100243 iterated over 56250 steps satisfies differential privacy with eps = 0.5 and delta = 8.333333333333334e-06.
Noise Scale:  1.2805631372100243
**********
Num Epochs: 30
tensor(1.8037, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.854366660118103
*************************
**** on testing set *****
Accuracy Rate: 0.8474440574645996
*************************
tensor(1.9189, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8164, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6348, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7480, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6523, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6172, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5361, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8398, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6152, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6543, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5322, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5664, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8926, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4580, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5215, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7148, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7891, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5449, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.898900032043457
*************************
**** on testing set *****
Accuracy Rate: 0.8893769979476929
*************************
tensor(1.8281, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3955, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6621, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9160, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4902, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5449, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4590, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6074, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3828, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7285, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5771, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4824, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6631, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4277, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5977, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6533, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5879, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9052166938781738
*************************
**** on testing set *****
Accuracy Rate: 0.8959664106369019
*************************
tensor(1.6094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5742, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6309, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3965, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7676, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3525, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7402, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6875, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5254, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4805, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4023, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5879, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5996, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6973, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6465, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5756.691421508789
Accuracy Rate: 0.8972643613815308
------------------------------------
NEW TEMPLATE
a blurry photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  0.5
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 1.2805631372100243 iterated over 56250 steps satisfies differential privacy with eps = 0.5 and delta = 8.333333333333334e-06.
Noise Scale:  1.2805631372100243
**********
Num Epochs: 30
tensor(1.9639, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.857450008392334
*************************
**** on testing set *****
Accuracy Rate: 0.8506389856338501
*************************
tensor(1.5938, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6016, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6182, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5781, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6406, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4980, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5830, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4160, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3242, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8311, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9746, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3770, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7324, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5840, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7070, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7617, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7793, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3965, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.8976166844367981
*************************
**** on testing set *****
Accuracy Rate: 0.8919728398323059
*************************
tensor(1.7363, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8223, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4805, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.2988, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5586, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5391, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3457, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3750, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6787, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8164, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7393, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6152, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6670, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9047333598136902
*************************
**** on testing set *****
Accuracy Rate: 0.8948681950569153
*************************
tensor(1.5156, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6240, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6602, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5020, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7256, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5215, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6201, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5312, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5186, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5205, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5703, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6172, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5078, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6484, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3623, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6562, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4492, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5794.044305324554
Accuracy Rate: 0.8950678706169128
------------------------------------
NEW TEMPLATE
a black and white photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  0.5
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 1.2805631372100243 iterated over 56250 steps satisfies differential privacy with eps = 0.5 and delta = 8.333333333333334e-06.
Noise Scale:  1.2805631372100243
**********
Num Epochs: 30
tensor(1.8096, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8543500304222107
*************************
**** on testing set *****
Accuracy Rate: 0.8461461663246155
*************************
tensor(1.6211, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6270, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7266, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7275, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6035, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3984, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5391, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3848, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5781, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7910, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6738, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8604, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4980, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3701, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5723, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8164, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7148, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.8966833353042603
*************************
**** on testing set *****
Accuracy Rate: 0.884085476398468
*************************
tensor(1.6367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5605, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6318, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7207, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6182, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5400, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6758, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4883, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4824, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7090, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6348, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5234, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5938, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7393, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4062, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3984, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4873, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9040499925613403
*************************
**** on testing set *****
Accuracy Rate: 0.8917731642723083
*************************
tensor(1.4727, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5615, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5176, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5234, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6230, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4746, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5059, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7295, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6611, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6357, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8691, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3926, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4238, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5799.501296520233
Accuracy Rate: 0.8937699794769287
------------------------------------
NEW TEMPLATE
a photo of the clothing item {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  0.5
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 1.2805631372100243 iterated over 56250 steps satisfies differential privacy with eps = 0.5 and delta = 8.333333333333334e-06.
Noise Scale:  1.2805631372100243
**********
Num Epochs: 30
tensor(1.8984, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8550666570663452
*************************
**** on testing set *****
Accuracy Rate: 0.8486421704292297
*************************
tensor(1.8848, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7314, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4590, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7197, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6562, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5654, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4805, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6240, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6250, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7344, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7637, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6357, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7012, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7363, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4043, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9385, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3750, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.8961333632469177
*************************
**** on testing set *****
Accuracy Rate: 0.8890774846076965
*************************
tensor(1.5986, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6641, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4727, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4668, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7021, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5801, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5830, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6875, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6201, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6240, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4062, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7412, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6152, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3740, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4980, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5029, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8311, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4229, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9043333530426025
*************************
**** on testing set *****
Accuracy Rate: 0.8974640369415283
*************************
tensor(1.5117, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5850, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5967, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4512, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4180, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7021, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7227, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4355, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4961, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5215, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5293, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5801, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6572, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6289, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4590, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4492, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5815.444615125656
Accuracy Rate: 0.8927715420722961
------------------------------------
