Certifying Distributional Robustness using Lipschitz Regularisation

Zac Cranko, Zhan Shi, Xinhua Zhang, Simon Kornblith, Richard Nock

Keywords: kernel method, adversarial learning, distributionally robust optimization

Abstract: Distributional robust risk (DRR) minimisation has arisen as a flexible and effective framework for machine learning. Approximate solutions based on dualisation have become particularly favorable in addressing the semi-infinite optimisation, and they also provide a certificate of the robustness for the worst-case population loss. However existing methods are restricted to either linear models or very small perturbations, and cannot find the globally optimal solution for restricted nonlinear models such as kernel methods. In this paper we resolved these limitations by upper bounding DRRs with an empirical risk regularised by the Lipschitz constant of the model, including deep neural networks and kernel methods. As an application, we showed that it also provides a certificate for adversarial training, and global solutions can be achieved on product kernel machines in polynomial time.

Code: https://github.com/learndeep2019/CertifyDRR.git