QMoE+: Hybrid Quantum Mixture of Experts

Abstract: Quantum mixture of experts (QMoE) extends conditional computation to the NISQ setting by distributing learning across parameterized quantum circuit (PQC) experts selected via a routing mechanism. Existing approaches are limited by single-block experts, lack of load balancing, and aggregation schemes that ignore routing amplitudes. We propose QMoE+, which uses two-block data re-uploading experts with learnable offsets, a coherent aggregation circuit over the joint routing-data Hilbert space, and a Switch-style load-balancing loss. Under top-k=1 sparse routing, QMoE+ activates only ∼28% of its parameters per inference while achieving consistent accuracy gains across seven datasets and four gate sets, winning 27/28 configurations with a mean improvement of +5.11% in the noiseless setting and +4.71% under depolarising noise. A decomposed ablation further shows that quantum coherence in the aggregation circuit outperforms an incoherent baseline in all seven datasets under p=0.01 noise (mean +1.80%), establishing an independent contribution beyond learnable aggregation parameters alone. Ablations confirm that load balancing is consistently beneficial, while data re-uploading provides the largest gains on complex tasks. Code is available at https://anonymous.4open.science/r/qmoe-plus/.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Junpei_Komiyama1

Submission Number: 8802