Indirect Supervision to Mitigate Perturbations

Mayank Kumar Kundalwal, Azad Singh, Deepak Mishra

Keywords: Indirect supervision, Perturbation, Downstream models, Image enhancement

Abstract: Vulnerability of state-of-the-art computer vision models to image perturbations has drawn considerable attention recently. Often these perturbations are imperceptible to humans because they target the perception of deep neural networks (DNNs) employed in the corresponding computer vision task. Recent studies have revealed that DNNs, which are unable to handle targeted perturbation often fail to handle untargeted perturbations as well such as Gaussian noise. Various techniques in past have been explored to mitigate both these types of perturbations ranging from classical preprocessing to current supervised and self-supervised deep discriminative and generative models. However, a common challenge with most of these techniques is that they approach the problem from a quality enhancement point of view, which is primarily driven by human perception. In addition, the supervised models require a large volume of gold standard unperturbed data, whereas others fail to take into account the feedback of the targeted downstream DNN. We propose to model this problem in indirect supervision framework, where we assume that the gold standard data is missing, however, a variable dependent on it is available and the dependency of the observed variable is stated by the considered downstream DNN. The proposed method maintains the advantages of supervised models while relaxing the requirement of gold standard unperturbed data. To prove its utility, we conduct several experiments on various network architectures for downstream tasks of classification and medical image segmentation. We used MNIST, CIFAR-10-C and ISIC skin lesion dataset in our experiments. In all the experiments, a considerable restoration in the performance of the considered downstream model is observed.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: This work reveals the promising performance of sparse indirect supervision in removing targeted and untargeted image perturbations in absence of gold standard clean images and information about the perturbation characteristics.

Reviewed Version (pdf): https://openreview.net/references/pdf?id=t5-aZtrb3D