ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients

Alessandro Cappelli, Ruben Ohana, Julien Launay, Laurent Meunier, Iacopo Poli

Keywords: adversarial attacks, adversarial defense, alternative training methods, direct feedback alignment, optical computing, fine-tuning

TL;DR: We find that the adversarial robustness of already robust models can be increased at no cost in natural accuracy through finetuning with a photonic processor and synthetic gradients.

Abstract: Robustness to adversarial attacks is typically obtained through expensive adversarial training with Projected Gradient Descent. We introduce ROPUST, a remarkably simple and efficient method to leverage robust pre-trained models and further increase their robustness, at no cost in natural accuracy. Our technique relies on the use of an Optical Processing Unit (OPU), a photonic co-processor, and a fine-tuning step performed with Direct Feedback Alignment, a synthetic gradient training scheme. We test our method on nine different models against four attacks in RobustBench, consistently improving over state-of-the-art performance. We also introduce phase retrieval attacks, specifically designed to target our own defense. We show that even with state-of-the-art phase retrieval techniques, ROPUST is effective.