NEW TEMPLATE
a photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 64
Optimizer Parameters: lr=1e-06, momentum=0.9, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  3.0
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.128% and noise_multiplier = 0.697094103073309 iterated over 23438 steps satisfies differential privacy with eps = 3 and delta = 1e-05.
Noise Scale:  0.697094103073309
**********
Num Epochs: 30
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.9285685420036316
*************************
**** on testing set *****
Accuracy Rate: 0.9226711988449097
*************************
tensor(2.2148, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1406, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0801, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0547, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0605, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1641, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9626159071922302
*************************
**** on testing set *****
Accuracy Rate: 0.9517316818237305
*************************
tensor(2.0566, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.2422, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9785, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0332, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0391, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0586, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0117, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0977, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9680107235908508
*************************
**** on testing set *****
Accuracy Rate: 0.9543192982673645
*************************
tensor(2., device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1562, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9707, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1152, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1172, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  4018.1382608413696
Accuracy Rate: 0.956210196018219
------------------------------------
NEW TEMPLATE
a blurry photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 64
Optimizer Parameters: lr=1e-06, momentum=0.9, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  3.0
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.128% and noise_multiplier = 0.697094103073309 iterated over 23438 steps satisfies differential privacy with eps = 3 and delta = 1e-05.
Noise Scale:  0.697094103073309
**********
Num Epochs: 30
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.923233687877655
*************************
**** on testing set *****
Accuracy Rate: 0.9184912443161011
*************************
tensor(2.3477, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.3105, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.2480, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1230, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1523, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.2109, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9618765711784363
*************************
**** on testing set *****
Accuracy Rate: 0.9496417045593262
*************************
tensor(2.0625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0117, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1289, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9863, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0977, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0293, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0078, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9688, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9674312472343445
*************************
**** on testing set *****
Accuracy Rate: 0.9536226391792297
*************************
tensor(2.0039, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0195, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1836, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9805, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0879, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0117, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9609, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  3906.647666454315
Accuracy Rate: 0.956011176109314
------------------------------------
NEW TEMPLATE
a black and white photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 64
Optimizer Parameters: lr=1e-06, momentum=0.9, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  3.0
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.128% and noise_multiplier = 0.697094103073309 iterated over 23438 steps satisfies differential privacy with eps = 3 and delta = 1e-05.
Noise Scale:  0.697094103073309
**********
Num Epochs: 30
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.9258911609649658
*************************
**** on testing set *****
Accuracy Rate: 0.9193869829177856
*************************
tensor(2.2070, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.2617, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1680, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1484, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0508, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1602, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9616568088531494
*************************
**** on testing set *****
Accuracy Rate: 0.9505374431610107
*************************
tensor(2.1289, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1133, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0586, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0117, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0898, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0332, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1562, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.967531144618988
*************************
**** on testing set *****
Accuracy Rate: 0.9545183181762695
*************************
tensor(1.9531, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1367, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0742, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0547, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0254, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9434, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9590, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  3913.2342987060547
Accuracy Rate: 0.9555135369300842
------------------------------------
NEW TEMPLATE
a bad photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 64
Optimizer Parameters: lr=1e-06, momentum=0.9, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  3.0
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.128% and noise_multiplier = 0.697094103073309 iterated over 23438 steps satisfies differential privacy with eps = 3 and delta = 1e-05.
Noise Scale:  0.697094103073309
**********
Num Epochs: 30
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.9294077754020691
*************************
**** on testing set *****
Accuracy Rate: 0.9229697585105896
*************************
tensor(2.2266, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1680, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0332, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1055, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1641, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0488, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1250, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1211, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9617367386817932
*************************
**** on testing set *****
Accuracy Rate: 0.9505374431610107
*************************
tensor(2.0918, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9980, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9902, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1016, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0254, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0918, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1035, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.9676910042762756
*************************
**** on testing set *****
Accuracy Rate: 0.9552149772644043
*************************
tensor(2.0176, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0078, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1426, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0703, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0625, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0586, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0820, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  3900.345216035843
Accuracy Rate: 0.9569068551063538
------------------------------------
NEW TEMPLATE
a good photo of a {}.
Files already downloaded and verified
Dataset: CIFAR10
Device: cuda
Batch Size: 64
Optimizer Parameters: lr=1e-06, momentum=0.9, weight_decay=1e-06
Classes: ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
Files already downloaded and verified
Files already downloaded and verified
Epsilon:  3.0
Delta:  1e-05
Clip Param C:  1.0
DP-SGD with sampling rate = 0.128% and noise_multiplier = 0.697094103073309 iterated over 23438 steps satisfies differential privacy with eps = 3 and delta = 1e-05.
Noise Scale:  0.697094103073309
**********
Num Epochs: 30
****the 0^th epoch *****
**** on training set *****
Accuracy Rate: 0.931805431842804
*************************
**** on testing set *****
Accuracy Rate: 0.9251592755317688
*************************
tensor(2.2148, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1758, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0977, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1836, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1289, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.3359, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0469, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1094, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 10^th epoch *****
**** on training set *****
Accuracy Rate: 0.9621763229370117
*************************
**** on testing set *****
Accuracy Rate: 0.9508360028266907
*************************
tensor(2.0859, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0254, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1465, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0195, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0801, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1035, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0938, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9727, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
****the 20^th epoch *****
**** on training set *****
Accuracy Rate: 0.968350350856781
*************************
**** on testing set *****
Accuracy Rate: 0.9550159573554993
*************************
tensor(2.0078, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9512, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.1445, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9961, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(1.9961, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0586, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
tensor(2.0137, device='cuda:0', dtype=torch.float16, grad_fn=<DivBackward0>)
Training Time:  3902.954886198044
Accuracy Rate: 0.9566082954406738
------------------------------------
