Revisiting Distribution Reconstruction via Sample Adaptability: Diffusion-Guided Data-Free Quantization

Biao Qian, Yang Wang, Zeqian Yi, Haipeng Liu, Jungong Han, Meng Wang

Keywords: Data-free quantization; distribution reconstruction; sample adaptability; diffusion models;

Abstract: Data-Free Quantization (DFQ), which seeks to align a low-bit quantized network Q with its full-precision counterpart P without access to original training data, has attracted growing attention. The core idea is to synthesize reconstructed samples that approximate the underlying distribution of real data. However, we observe that reconstructed samples in existing arts are adaptive only to Q with specific bit widths, rather than all bit widths, especially ultra-low bit widths. This raises a key challenge: how to rectify distributional information during sample reconstruction to produce desirable samples that generalize across varied quantization levels. In this paper, we revisit distribution reconstruction via sample adaptability, revealing that 1) the desirable samples enjoy benefits of rectifying distribution reconstruction via adaptability information; beyond that, 2) the forward diffusion process is an optimal noise strategy to rectify reconstructed samples for obtaining desirable samples; and propose a novel Diffusion-Guided Data-Free Quantization approach, dubbed DiffDFQ. Unlike conventional direct optimization, we rectify distribution reconstruction via noise diffusion in a progressive approximation manner. Technically, we decompose DFQ into three stages: sample synthesis to obtain reconstructed samples; upon that, sample diffusion to progressively infuse the noise via forward diffusion process, yielding desirable samples for varied Q; and network calibration to calibrate Q with progressive selection strategy over a series of diffused samples. Our DiffDFQ enjoys the appealing insights: 1) the diffused samples exhibit effective balance between distribution reconstruction and sample adaptability to facilitate varied Q, especially ultra-low bit widths, e.g., 2 bits and 3 bits; notably, 2) unlike the generator-reliance arts requiring up to 1.2M synthetic samples, DiffDFQ synthesizes merely 5.12K (1K) samples to earn performance gain over ImageNet for classification. Our empirical studies verify the merits of DiffDFQ over state-of-the-arts for classification across varied bits to Q. Our code is available in the supplementary material package.

Primary Area: learning theory

Submission Number: 8490