Feature Discrimination Analysis for Binary and Ternary Quantization
Keywords: binary quantization, ternary quantization, feature quantization, discriminant analysis, sparse representation
Abstract: In machine learning, quantization is widely used to simplify data representation and facilitate algorithm deployment on hardware. Considering the fundamental role of classification in machine learning, it is imperative to investigate the impact of quantization on classification. Current research primarily revolves around quantization errors, under the assumption that higher quantization errors generally lead to lower classification performance. However, this assumption lacks a solid theoretical foundation, and often contradicts empirical findings. For instance, some extremely low bit-width quantization methods, such as $\{0,1\}$-binary quantization and $\{0, \pm1\}$-ternary quantization, can achieve comparable or even superior classification accuracy compared to the original non-quantized data, despite exhibiting high quantization errors. To evaluate the classification performance more accurately, we propose to directly investigate the feature discrimination of quantized data, rather than analyze its quantization error. It is found that binary and ternary quantization can surprisingly improve, rather than degrade, the feature discrimination of original data. This remarkable performance is validated through classification experiments on diverse data types, including images, speech and text.
Primary Area: other topics in machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7214
Loading