Empirically Investigating the Trade-Offs in Deterministic Certified Training

ICLR 2026 Conference Submission644 Authors

Keywords: Certified Training, Neural Network Verification, Hyperparameter Optimisation, Robustness

TL;DR: We propose, for the first time, an automated hyperparameter optimisation technique for certified training, thereby establishing a new state-of-the-art as well as a novel evaluation mechanism based on Pareto fronts.

Abstract: While there have been numerous advancements regarding the performance of deep neural networks on a broad range of supervised learning tasks, their adversarial robustness remains a major concern. To mitigate this, neural network verification aims to provide mathematically rigorous robustness guarantees at the cost of substantial computational requirements. Certified training}methods overcome this challenge by optimising for verifiable robustness during training, which, however, usually results in substantial decrease of performance on clean data. This robustness-accuracy trade-off has been extensively studied in the context of adversarial training but remains mostly unexplored for certified training. To control this trade-off, certified training techniques expose hyperparameters, which, to date, have been manually tuned to one specific configuration that compares favourable to the previous state-of-the-art. In this work, we present a novel fully-automated hyperparameter optimisation procedure for certified training that yields a Pareto front of optimal configurations with regard to the robustness-accuracy trade-off. Our approach facilitates the fair, principled and nuanced comparison of the performance of different methods. We show that most methods yield better trade-offs than previously assumed, thereby establishing a new state of the art in certified training of deep neural networks. In addition, we demonstrate that performance improvements reported over recent years are far less pronounced when all methods have been carefully tuned.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 644