BLADE: Binary Learning via Algebraic Dual Estimation for the Exact Edge of Stability in 1-Bit Networks

Şuayp Talha Kocabay, Talha Rüzgar Akkuş

Keywords: BLADE, Edge of Stability, 1-bit Quantization, Binary Neural Networks, Forward- Mode AD, Dual Numbers, Surrogate Gradients

TL;DR: We propose BLADE, a backprop-free framework for 1-bit networks using dual-number JVPs to achieve O(1) activation memory, providing the first empirical evidence that the Edge-of-Stability persists in discontinuous binary architectures.

Abstract: The *Edge-of-Stability* (EoS) phenomenon---whereby the top Hessian eigenvalue $\lambda_{\max}$ stabilizes near $2/\eta$ during gradient descent---is well-documented for smooth networks but theoretically unclear for 1-bit activations. We propose **BLADE** (**B**inary **L**earning via **A**lgebraic **D**ual **E**stimation), a backprop-free framework that embeds surrogate directional derivatives into the forward pass via custom Jacobian-vector products (JVPs) on dual numbers. This yields exact primal binarization and $O(1)$ activation memory in network depth. Evaluated across five benchmarks against five backprop surrogates, BLADE achieves state-of-the-art results, including $100\%$ accuracy on Wine. We provide the first empirical evidence that EoS persists in discontinuous 1-bit networks, despite the characteristic curvature overshoot induced by the non-smooth geometry.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 6