Unforgeability in Stochastic Gradient Descent

Teodora Baluta, Ivica Nikolic, Racchit Jain, Divesh Aggarwal, Prateek Saxena

Abstract:

Stochastic Gradient Descent (SGD) is a popular training algorithm,
a cornerstone of modern machine learning systems. Several security applications benefit from determining if SGD executions are
forgeable, i.e., whether the model parameters seen at a given step
are obtainable by more than one distinct set of data samples. In
this paper, we present the first attempt at proving impossibility of
such forgery. We furnish a set of conditions, which are efficiently
checkable on concrete checkpoints seen during training runs, under
which checkpoints are provably unforgeable at that step. Our experiments show that the conditions are somewhat mild and hence
always satisfied at checkpoints sampled in our experiments. Our
results sharply contrast prior findings at a high level: We show that
checkpoints we find to be provably unforgeable have been deemed
to be forgeable using the same methodology and experimental setup
suggested in prior work. This discrepancy arises because of unspecified subtleties in definitions. We experimentally confirm that the
distinction matters, i.e., small errors amplify during training to produce significantly observable difference in final models trained. We
hope our results serve as a cautionary note on the role of algebraic
precision in forgery definitions and related security arguments.