On the Learnability of Distribution Classes with Adaptive Adversaries

Tosca Lechner, Alex Bie, Gautam Kamath

Abstract: We consider the question of learnability of distribution classes in the presence of adaptive adversaries -- that is, adversaries capable of intercepting the samples requested by a learner and applying manipulations with full knowledge of the samples before passing it on to the learner. This stands in contrast to oblivious adversaries, who can only modify the underlying distribution the samples come from but not their i.i.d.\ nature. We formulate a general notion of learnability with respect to adaptive adversaries, taking into account the budget of the adversary. We show that learnability with respect to additive adaptive adversaries is a strictly stronger condition than learnability with respect to additive oblivious adversaries.

Lay Summary:

Generalizing from training data underlies most machine learning processes. Often this training data is assumed to be generated directly from the phenomena one wants to learn. In our work we study the situation, where an adversary gets to manipulate the training data, before the learner gets to see it. We study adaptive adversaries, who have access to the whole training data and can therefore manipulate with this full knowledge. We contrast them with oblivious adversaries, who only are aware of the data generating process, but not of the training data itself. We show that adaptive adversaries can be strictly stronger than oblivious adversaries.
In particular, we study additive adversaries, who can add data points and subtractive adversaries, who can delete data points. We show a separation between adaptive additive and oblivious adaptive adversaries. Thus, we show that in some situations adding data points when knowing a sample can gravely hurt the learning process, while similar additive manipulations on the data-generating process will not hurt the learning process too much.

Primary Area: Theory->Learning Theory

Keywords: distribution learning, robustness

Submission Number: 1664