ZeroP: Zero-Shot Quantization via Proxy Data

GuangYan Zhang, Hongmin Xu, Zhitong Zheng, Haifeng Liu

Primary Area: infrastructure, software libraries, hardware, etc.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Zero-Shot; Quantization; Data-Free; Proxy Data;

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Zero-shot quantization (ZSQ) is a promising approach for achieving low-bit constraint networks without relying on the original data (OD). However, due to the high cost and privacy concerns associated with OD, it is often scarce, leading to the unsatisfactory performance of ZSQ. Most ZSQ methods rely solely on synthetic data (SD) to mitigate this issue. In this paper, we propose a novel ZSQ framework, named ZeroP, that leverages publicly available data - proxy data (PD) - as a substitute for the OD. We first explore the impact of PD on the performance of current ZSQ methods over 16 different computer vision datasets and introduce a simple and effective PD selection method based on batch-normalization statistics(BNS) to select the optimal PD. We then apply ZeroP to three state-of-the-art pure-SD (using only SD) methods, achieving 7% to 16% improvements in accuracy for MobileNetV1 on ImageNet-1K in a 4-bit setting. Furthermore, we demonstrate the effectiveness of ZeroP on extensive models and datasets. For example, ZeroP achieves a top-1 accuracy of 72.17% for ResNet-50 on ImageNet-1K in a 4-bit setting, outperforming the SOTA pure-SD method by 3.9%. Overall, our results indicate that ZeroP offers a promising solution for achieving high-performance low-bit networks without relying on original training data and opens up new avenues for using publicly available data for data-free tasks.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 819