Gauge Symmetries for Efficient Zero- Knowledge Proofs of Transformers

Hong Wang, Kelly Wang

Keywords: Transformer gauge symmetry, Zero-knowledge machine learning (ZKML), RoPE/GQA/MQA/MOE, Halo2 / PLONKish circuit optimization

TL;DR: GaugeZKP leverages attention’s gauge symmetry to cut ZK prover cost. One-time PoGE certifies canonical weights; PoVI verifies runs. RoPE and GQA/MQA/MoE compatible; reductions without changing model behavior

Abstract:

We introduce GaugeZKP, a symmetry-aware verification framework for Transformers that exploits the maximal gauge group of attention. For canonical models the maximal group is Gₘₐₓ = ((GL(dₖ))ʰ × (GL(dᵥ))ʰ) ⋊ Sₕ; with RoPE (LLaMA/Qwen), the Q/K action reduces to the rotary commutant C_RoPE.

We operationalize this via a one-time proof of gauge equivalence (PoGE) to a canonical model and per-inference proofs (PoVI) on that canonical model. On Halo2 circuits, canonicalization reduces model-level prover gates/constraints by up to ≈26% without changing model function; because this optimization is upstream of the prover, pairing with frameworks like EZKL/zkVM further reduces proving time/memory on the smaller circuit. Analytically, the savings multiply with parameter tying in grouped/single-query attention (GQA/MQA) and with MoE sparsity, since PoGE/PoVI scale with the number of distinct parameter blocks rather than with the head count.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 2747