ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs

Yuchen Yang, Shubham Ugare, Yifan Zhao, Gagandeep Singh, Sasa Misailovic

Keywords: Quantization, Robustness, Certification, Formal Verification

Abstract:

Mixed precision quantization has become an important technique for
enabling the execution of deep neural networks (DNNs) on limited resource computing platforms.
Traditional quantization methods have primarily concentrated on maintaining
neural network accuracy, either ignoring the impact of quantization on the
robustness of the network, or using only empirical techniques for improving
robustness. In contrast, techniques for robustness certification, which can
provide strong guarantees about the robustness of DNNs have not been used
during quantization due to their high computation cost and/or scalability
issues.

This paper introduces ARQ, an innovative mixed-precision quantization method that not only
preserves the clean accuracy of the smoothed classifiers but also maintains
their certified robustness. ARQ uses reinforcement learning to find accurate and robust
DNN quantization, while efficiently leveraging randomized smoothing,
a popular class of statistical DNN verification algorithms, to guide the search process.
We compare ARQ with multiple state-of-the-art quantization techniques on
several DNN architectures commonly used in quantization studies: ResNet-20 on
CIFAR-10, ResNet-50 on ImageNet, and MobileNetV2 on ImageNet.
We demonstrate that ARQ consistently performs better than these baselines
across all the benchmarks and the input perturbation levels. In many cases, the performance of ARQ quantized networks can reach that of the original DNN with floating-point weights, but with only 1.5% instructions.

Primary Area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8787