A Numerical Study of Robustness Verification for Lightning Self-Attention

Yulia Alexandr, Hao Duan, Guido Montufar

Keywords: ED degree, ED discriminant, robustness verification, decision boundary, polynomial networks, transformer

Abstract: We study the algebraic complexity of robustness verification for lightning self-attention. Since lightning self-attention is polynomial, a binary linear readout defines a cubic decision boundary, and exact Euclidean robustness certification reduces to a nearest-boundary problem. We use the Euclidean distance degree to measure boundary complexity and the ED discriminant to study how the real nearest-boundary problem varies with the input. Our computations show that lightning self-attention boundaries have ED degree far below that of generic cubic hypersurfaces, while still varying systematically with sequence length, embedding dimension, output dimension, and attention rank. We also compute two-dimensional slices of the ED discriminant for a small architecture, showing chambers with different numbers of real critical points.

Paper Type: Short (4 pages)

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 75