Quantum-Inspired Complex Transformers: Resolving the Fundamental Algebraic Ambiguity for Enhanced Neural Representations

NeurIPS 2025 Workshop NeurReps Submission1 Authors

Keywords: Quantum-Inspired Computing, Complex Neural Networks, Algebraic and Geometric Deep Learning, Parameter Efficiency, Transformers

TL;DR: QIC Transformers learn imaginary numbers via "quantum superposition," reducing parameters by ~21% & improving accuracy in sequence tasks. Training is ~2x slower, but the efficiency suits constrained deployments.

Abstract: We present Quantum-Inspired Complex (QIC) Transformers, a novel architecture that enhances neural network expressiveness through learnable algebraic structures. Our key insight is that the fundamental equation $x^2 = -1$ has two solutions, traditionally resolved by arbitrary selection. We propose treating the imaginary unit as a learnable quantum superposition: $J(\theta) = \cos(\theta)J_+ + \sin(\theta)J_-$, where $\theta$ is trainable. This yields $J^2 = -1 + \sin(2\theta)$, creating an adaptive algebra that interpolates between mathematical regimes. When integrated into Transformers, this approach achieves 98.50\% accuracy versus 97.75\% for standard models, while reducing parameters by 20.96\%. Despite a 2.17× training time increase, QIC Transformers offer compelling advantages for parameter-constrained deployments. We provide mathematical foundations, architectural specifications, and empirical validation demonstrating that learnable algebraic structures fundamentally enhance neural network capabilities.

Submission Number: 1