Hessian-aware Training for Enhancing DNN Resilience to Bitwise Corruptions in Their Parameters

Tahmid Hasan Pranto, Seijoon Kim, Lizhong Chen, Sanghyun Hong

Abstract:

Deep neural networks are not resilient to parameter corruptions: even a single-bitwise error in their parameters in memory can cause an accuracy drop of over 10%, and in the worst-cases, up to 99%. This susceptibility poses great challenges in deploying models on computing platforms, where adversaries can induce random/targeted bit-flips, e.g., through software-induced fault attacks like Rowhammer. Most prior work addresses this issue with hardware or system-level approaches, such as integrating additional hardware components to verify a model’s integrity at inference. However, these methods have not been widely deployed as they require infrastructure or platform-wide modifications.

In this paper, we propose a new approach to addressing this issue: training models to be more resilient to bitwise corruptions to their parameters. Our approach, Hessian-aware training, promotes models to learn flatter loss surfaces. We show that existing training methods designed to improve generalization (e.g., through sharpness-aware minimization) do not enhance resilience to parameter corruptions. In contrast, models trained with our method demonstrate improved resilience to parameter corruptions, particularly with a 20–50% reduction in the number of bits whose individual flipping leads to a 90–100% accuracy drop. We also characterize the factors that may influence this increased resilience. Moreover, we show the synergy between ours and existing hardware and system-level defenses.

Submission Type: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: We made minor camera-ready revisions to improve clarity and presentation. Specifically, we updated the manuscript from the anonymous review version to the final accepted version by adding author information and the OpenReview review link. We also expanded and reorganized the Background section into clearer subsections, including IEEE-754 floating-point representation, relation to prior curvature-aware/robustness methods, and bit-flip attacks on DNNs via Rowhammer. In addition, we added explanatory text to better clarify the intuition behind Hessian-aware training and why smoother loss landscapes improve resilience to parameter-level bit-flip corruptions. These revisions do not change the main method, experimental results, conclusions, or claims of the paper.

Assigned Action Editor: ~Jonathan_Ullman1

Submission Number: 5889