QIVPT: Quantum-Inspired Visual Prompt Tuning via Givens Rotations-Based Orthogonal Transformation

Yijun Wang, Qiyuan Zhuang, chunlin wen, Yu Zhang, Xiaogang Qiang, Xiu-Shen Wei

Keywords: Visual Prompt Tuning, Qunatum-Inspired, Orthogonal Transformation, Givens Rotations

Abstract: Prompt tuning has become a powerful and parameter-efficient approach for adapting pre-trained vision transformers to diverse downstream tasks. However, existing methods often lack structural interpretability, treating prompt tokens as free parameters without semantic constraints. Inspired by the principles of quantum computation, particularly superposition and unitary evolution, we propose a novel method called Quantum-Inspired Visual Prompt Tuning (QIVPT). By viewing prompt tokens as semantic analogues of quantum states in a high-dimensional Hilbert space, we design a structured and orthogonality-preserving transformation over prompts using learnable sequences of Givens rotations. This approach simulates quantum-like semantic evolution, enabling controllable and reversible interactions among prompt embeddings, while maintaining low parameter overhead and enhancing adaptability across diverse tasks. Unlike parameterized quantum circuits, our method is lightweight, fully differentiable, and hardware-friendly. Extensive experiments on VTAB-1k and FGVC benchmarks demonstrate that QIVPT consistently outperforms existing prompt tuning baselines, offering improved generalization with minimal additional parameters.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 5210