CERTIFIED VS. EMPIRICAL ADVERSARIAL ROBUSTNESS VIA HYBRID CONVOLUTIONS WITH ATTENTION STOCHASTICITY

ICLR 2026 Conference Submission20436 Authors

Keywords: Certified Defense, Empirical Defense, Adversarial Robustness

Abstract: We introduce Hybrid Convolutions with Attention Stochasticity (HyCAS), an adversarial defense that narrows the long-standing gap between provable robustness under ℓ2 certificates and empirical robustness against strong ℓ∞ attacks, while preserving strong generalization on both natural- and medical-image tasks. HyCAS unifies deterministic and randomized principles by coupling 1-Lipschitz, spectrally normalized convolutions with two stochastic components—spectral normalized random-projection filters and a randomized attention-noise mechanism. Injecting smoothing randomness inside the architecture yields an overall ≤ 2- Lipschitz network with formal certificates. Extensive experiments on diverse benchmarks—including CIFAR-10/100, ImageNet-1k and NIH Chest X-ray—show that HyCAS surpasses prior certified and empirical defenses, boosting certified accuracy by up to ≈ 7.3% and empirical robustness by up to ≈ 3.3%, without sacrificing clean accuracy. These results demonstrate that a hybrid deterministic–stochastic design can harmonize provable and empirical adversarial robustness, fostering safer deployment of deep models in high-stakes applications.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 20436