Lipschitz-Constrained Convolutional Layers Using Convex Projection

Bhartendu Kumar, Kunal N. Chaudhury

Abstract: The problem of training a convolutional neural network (CNN) with a stipulated Lipschitz bound comes up in applications such as adversarial robustness, stability of closed-loop controllers, and image reconstruction. The present work was motivated by Plug-and-Play (PnP) and Regularization-by-Denoising (RED) which use CNN denoisers for image reconstruction. It has been shown that the convergence of these iterative algorithms can be guaranteed by constraining the Lipschitz bound of the denoiser. We make the case that using a contractive CNN denoiser is a straightforward means to certify convergence. In particular, we show how a contractive CNN denoiser can be trained using convex projections within the paradigm of gradient-based learning and how the projection problem can be reduced to a tractable convex program. Apart from the theoretical guarantee, the regularization capacity of the trained denoiser is shown to be competitive with BM3D and DnCNN.