Distributional Adversarial Loss

Saba Ahmadi, Siddharth Bhandari, Avrim Blum, Chen Dan, Prabhav Jain

TL;DR: We introduce a new notion of adversarial loss, distributional adversarial loss, that unifies two lines of work, randomized smoothing methods and robust learning in the PAC-learning setting.

Abstract:

We initiate the study of a new notion of adversarial loss which we call distributional adversarial loss. In this notion, we assume for each original example, the allowed adversarial perturbation set is a family of distributions, and the adversarial loss over each example is the maximum loss over all the associated distributions. The goal is to minimize the overall adversarial loss. We show generalization guarantees for our notion of adversarial loss. Our notion of adversarial loss contrasts the prior work that considers a set of points, not distributions, as the perturbation set of each clean example. As an application of our approach, we show how to unify the two lines of work on randomized smoothing and robust learning in the PAC-learning setting and derive sample complexity bounds for randomized smoothing methods.

Furthermore, we investigate the role of randomness in achieving robustness against adversarial attacks. We show a general derandomization technique that preserves the extent of a randomized classifier's robustness against adversarial attacks and show its effectiveness empirically.

Submission Number: 769