Normalization-Free Knowledge Distillation for Variable-Bit CSI Feedback in Massive MIMO

Eunju Seo, WooJun Kim, Dongjin Ji

Keywords: Massive MIMO, CSI Feedback, Knowledge Distillation, Variable-Bit Feedback, Normalization-Free

TL;DR: A Normalization-Free Knowledge Distillation (NF-KD) method that preserves physical scale and global phase structure for variable-bit CSI feedback in Massive MIMO.

Abstract: Efficient Channel State Information (CSI) feedback is critical for beamforming in Frequency Division Duplexing (FDD) Massive MIMO, yet variable-bit deep learning schemes suffer from trailing-bit information loss under constrained training budgets~\citep{ji2024concrete}. While Knowledge Distillation (KD) is a natural remedy, we identify a previously overlooked problem: standard Z-score normalization applied during KD destroys the physical scale information embedded in wireless channel tensors, degrading reconstruction quality across variable feedback lengths. We propose Normalization-Free Knowledge Distillation (NF-KD), a training framework that preserves physical scale by applying Kullback--Leibler Divergence (KLD) directly to raw, temperature-scaled tensors without any normalization step. Experiments on the DeepMIMO dataset show that NF-KD achieves $-12.87$\,dB Normalized Mean Squared Error (NMSE) at 256 bits (vs.\ $-11.98$\,dB baseline), consistent cosine similarity gains across all feedback lengths, and a $0.056$\,dB reduction in beamforming gain loss that translates to $+0.019$\,bps/Hz spectral efficiency under Zero-Forcing (ZF) precoding at $20$\,dB Signal-to-Noise Ratio (SNR)---all within a 500-epoch training budget. Ablation confirms that normalization is the critical failure factor.

Submission Number: 15