DUAL-GDFQ: A Dual-Generator, Dual-Phase Learning Approach for Data-Free Quantization

Sihan Wang, Zhi Han, Xiyao Liu

Abstract: Data-free quantization (DFQ) seeks to maximize the performance of quantized networks without requiring original training data. Conventional methods, which use synthetic samples from generators for network fine-tuning, often yield inferior results compared to training conducted with real data. To mitigate this problem, we introduce a dual-generator, dual-phase learning generative data-free quantization (DUAL-GDFQ) method, which utilizes two generators: a knowledge-matching generator and a knowledge-promoting generator for replicating the original data distribution as well as keeping samples informative. Additionally, inspired by meta-learning, the proposed novel dual-phase learning scheme can effectively utilize the capabilities of both generators by aligning their gradient descent directions. Theoretical analysis and extensive experiments demonstrate that our method successfully minimizes performance degradation in quantized networks and can achieve performance levels comparable to training with real data.