NEW TEMPLATE
a photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  0.25
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 1.997240523781446 iterated over 56250 steps satisfies differential privacy with eps = 0.25 and delta = 8.333333333333334e-06.
Noise Scale:  1.997240523781446
**********
Num Epochs: 30
tensor(1.6523, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8515666723251343
*************************
**** on testing set *****
Accuracy Rate: 0.8422523736953735
*************************
tensor(1.8223, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6963, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6270, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6523, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7822, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7168, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5156, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7109, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6543, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5645, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5449, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5449, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7773, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7754, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5479, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6475, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5488, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.8788166642189026
*************************
**** on testing set *****
Accuracy Rate: 0.8705071806907654
*************************
tensor(1.5703, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5137, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7539, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8887, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6309, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6992, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6328, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5957, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8242, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5449, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5186, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5391, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8750, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0742, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7695, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6123, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7441, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3975, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.8702166676521301
*************************
**** on testing set *****
Accuracy Rate: 0.8652156591415405
*************************
tensor(1.6855, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8770, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8809, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8516, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6904, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7812, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8428, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6309, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6934, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5439, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5654, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7988, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6289, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6738, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4824, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5879, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6387, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5910.176423549652
Accuracy Rate: 0.860223650932312
------------------------------------
NEW TEMPLATE
a blurry photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  0.25
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 1.997240523781446 iterated over 56250 steps satisfies differential privacy with eps = 0.25 and delta = 8.333333333333334e-06.
Noise Scale:  1.997240523781446
**********
Num Epochs: 30
tensor(1.9570, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8472999930381775
*************************
**** on testing set *****
Accuracy Rate: 0.840355396270752
*************************
tensor(1.5039, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8604, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8828, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6807, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7109, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6875, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5742, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5117, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7129, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7109, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7568, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8027, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6855, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5391, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5283, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5801, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6025, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.8775833249092102
*************************
**** on testing set *****
Accuracy Rate: 0.8666133880615234
*************************
tensor(1.5107, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7246, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5898, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6836, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7803, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5322, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5605, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6426, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7480, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5391, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4746, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7949, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6387, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5762, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5684, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6191, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.8734666705131531
*************************
**** on testing set *****
Accuracy Rate: 0.8677116632461548
*************************
tensor(1.5195, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8555, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9336, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4561, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4023, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7305, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6523, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4004, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4785, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7109, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7461, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6475, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5381, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7598, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5969.5448207855225
Accuracy Rate: 0.8571285605430603
------------------------------------
NEW TEMPLATE
a black and white photo of a {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  0.25
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 1.997240523781446 iterated over 56250 steps satisfies differential privacy with eps = 0.25 and delta = 8.333333333333334e-06.
Noise Scale:  1.997240523781446
**********
Num Epochs: 30
tensor(1.9883, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8500666618347168
*************************
**** on testing set *****
Accuracy Rate: 0.8425518870353699
*************************
tensor(1.7188, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6406, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7051, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7012, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7461, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7188, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6992, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8203, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9102, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7656, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5156, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4600, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7070, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6504, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5068, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7900, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.8794833421707153
*************************
**** on testing set *****
Accuracy Rate: 0.8703075051307678
*************************
tensor(1.7217, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6797, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4727, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6797, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6953, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6650, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6855, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6143, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5059, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7832, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5908, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4854, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8594, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5586, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6611, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4307, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5508, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.8729000091552734
*************************
**** on testing set *****
Accuracy Rate: 0.865914523601532
*************************
tensor(1.6465, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9912, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7285, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5049, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6934, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7549, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7559, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7344, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5430, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6836, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5723, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7129, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5898, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7441, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6387, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5449, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8125, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  6020.108610391617
Accuracy Rate: 0.8603234887123108
------------------------------------
NEW TEMPLATE
a photo of the clothing item {}.
Dataset: Fashion MNIST
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, momentum=0.9, weight_decay=1e-06
Classes: ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']
Num Epochs:30
Epsilon:  0.25
Delta:  8.333333333333334e-06
Clip Param C:  0.1
DP-SGD with sampling rate = 0.0533% and noise_multiplier = 1.997240523781446 iterated over 56250 steps satisfies differential privacy with eps = 0.25 and delta = 8.333333333333334e-06.
Noise Scale:  1.997240523781446
**********
Num Epochs: 30
tensor(1.6367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.8536999821662903
*************************
**** on testing set *****
Accuracy Rate: 0.844648540019989
*************************
tensor(1.7402, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5967, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6348, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5898, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8613, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7080, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8164, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5059, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7148, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4072, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4854, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4414, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4414, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6172, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8223, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6426, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6426, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.8772667050361633
*************************
**** on testing set *****
Accuracy Rate: 0.8663138747215271
*************************
tensor(1.8291, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5518, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7588, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5703, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6807, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6719, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7812, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6680, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6152, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4688, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5713, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4551, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8027, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6621, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7012, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7451, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8926, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7051, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.8720499873161316
*************************
**** on testing set *****
Accuracy Rate: 0.8612220287322998
*************************
tensor(1.6016, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6309, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6738, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5840, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7803, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5596, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5684, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4785, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6582, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5615, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7988, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8184, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6016, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7754, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7441, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5049, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  5982.334321975708
Accuracy Rate: 0.8547323942184448
------------------------------------
