NEW TEMPLATE
a photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, betas=(0.9, 0.98), eps=1e-06, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  0.25
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.064% and noise_multiplier = 2.1434173513415473 iterated over 46875 steps satisfies differential privacy with eps = 0.25 and delta = 1e-05.
Noise Scale:  2.1434173513415473
**********
Num Epochs: 30
tensor(1.9062, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.9255038499832153
*************************
**** on testing set *****
Accuracy Rate: 0.9216253757476807
*************************
tensor(1.6035, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4961, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5996, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5332, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6162, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4316, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4834, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5518, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6387, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6270, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5547, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4600, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3164, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5928, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6162, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9433581233024597
*************************
**** on testing set *****
Accuracy Rate: 0.9386980533599854
*************************
tensor(1.5527, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5273, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4375, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6348, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5000, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5439, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3516, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5352, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5527, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4805, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5000, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4268, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5537, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6270, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4258, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9410188794136047
*************************
**** on testing set *****
Accuracy Rate: 0.9330071806907654
*************************
tensor(1.4277, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3848, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3760, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4805, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4668, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4570, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6895, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4385, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5195, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6133, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4746, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5146, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4336, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7695, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  8913.151130914688
Accuracy Rate: 0.9333066940307617
------------------------------------
NEW TEMPLATE
a blurry photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, betas=(0.9, 0.98), eps=1e-06, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  0.25
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.064% and noise_multiplier = 2.1434173513415473 iterated over 46875 steps satisfies differential privacy with eps = 0.25 and delta = 1e-05.
Noise Scale:  2.1434173513415473
**********
Num Epochs: 30
tensor(1.8711, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.9192258715629578
*************************
**** on testing set *****
Accuracy Rate: 0.9146365523338318
*************************
tensor(1.8184, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7793, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4883, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8691, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5908, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4932, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4463, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3691, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5527, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5000, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3711, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3438, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6543, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3691, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9436180591583252
*************************
**** on testing set *****
Accuracy Rate: 0.9374001622200012
*************************
tensor(1.5225, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6611, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4258, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4863, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3154, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3281, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4707, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5947, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4902, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6816, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4785, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4424, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5859, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4277, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9436180591583252
*************************
**** on testing set *****
Accuracy Rate: 0.9382987022399902
*************************
tensor(1.4785, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6309, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6328, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.2969, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6191, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4883, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3789, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5879, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4336, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6172, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6230, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5117, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4902, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  9072.382814407349
Accuracy Rate: 0.9334065318107605
------------------------------------
NEW TEMPLATE
a black and white photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, betas=(0.9, 0.98), eps=1e-06, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  0.25
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.064% and noise_multiplier = 2.1434173513415473 iterated over 46875 steps satisfies differential privacy with eps = 0.25 and delta = 1e-05.
Noise Scale:  2.1434173513415473
**********
Num Epochs: 30
tensor(1.6758, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.9253638982772827
*************************
**** on testing set *****
Accuracy Rate: 0.9212260246276855
*************************
tensor(1.9688, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7012, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4941, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5059, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3574, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6289, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4404, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4541, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4004, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4912, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7607, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4707, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5742, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4160, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4395, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9438979625701904
*************************
**** on testing set *****
Accuracy Rate: 0.9378993511199951
*************************
tensor(1.8027, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6064, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4424, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5156, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4707, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6855, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4473, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4365, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4150, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4111, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5059, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5918, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3750, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6230, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9424384236335754
*************************
**** on testing set *****
Accuracy Rate: 0.9347044825553894
*************************
tensor(1.5176, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5166, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5186, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3594, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3457, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4727, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5586, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5273, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4385, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4336, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4277, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  9355.943542957306
Accuracy Rate: 0.9323083162307739
------------------------------------
NEW TEMPLATE
a bad photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, betas=(0.9, 0.98), eps=1e-06, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  0.25
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.064% and noise_multiplier = 2.1434173513415473 iterated over 46875 steps satisfies differential privacy with eps = 0.25 and delta = 1e-05.
Noise Scale:  2.1434173513415473
**********
Num Epochs: 30
tensor(1.5654, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.9281829595565796
*************************
**** on testing set *****
Accuracy Rate: 0.922723650932312
*************************
tensor(1.4531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7461, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5918, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4297, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6797, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6484, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5781, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6104, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5498, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6719, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4902, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6387, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5498, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3379, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6211, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4795, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9428982734680176
*************************
**** on testing set *****
Accuracy Rate: 0.9356030225753784
*************************
tensor(1.7041, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5781, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3350, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3408, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4326, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6211, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5713, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6562, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3379, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8457, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4141, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3867, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3672, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9408789277076721
*************************
**** on testing set *****
Accuracy Rate: 0.934904158115387
*************************
tensor(1.5801, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5293, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5645, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3994, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4961, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5781, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3350, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4180, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4512, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4023, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4326, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3301, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.8555, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5879, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  9333.099712371826
Accuracy Rate: 0.931908905506134
------------------------------------
NEW TEMPLATE
a good photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 32
Optimizer Parameters: lr=1e-05, betas=(0.9, 0.98), eps=1e-06, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  0.25
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.064% and noise_multiplier = 2.1434173513415473 iterated over 46875 steps satisfies differential privacy with eps = 0.25 and delta = 1e-05.
Noise Scale:  2.1434173513415473
**********
Num Epochs: 30
tensor(1.5723, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.9280830025672913
*************************
**** on testing set *****
Accuracy Rate: 0.9256190061569214
*************************
tensor(1.9277, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5234, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5879, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4629, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3848, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9971, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7383, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5645, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4180, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4551, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5264, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4727, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4336, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4102, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4365, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9462971687316895
*************************
**** on testing set *****
Accuracy Rate: 0.9399960041046143
*************************
tensor(1.5430, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4121, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4775, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5703, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4648, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4365, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6455, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4688, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7061, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5537, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4180, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5332, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.6953, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5898, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9453774690628052
*************************
**** on testing set *****
Accuracy Rate: 0.9387978911399841
*************************
tensor(1.3008, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5254, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4844, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.3984, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5674, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4961, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4766, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4307, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4570, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.5137, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4863, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4785, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.4043, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.7285, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  9254.21479177475
Accuracy Rate: 0.9351038336753845
------------------------------------
